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ABSTRACT 

One  way  of  comparing  alternative  deployment  plans  is  to  examine  how  combat 
aircraft  tasked  within  a  deployment  plan  meet  the  combat  commander’s  requirements. 
Both  the  absolute  capabilities  and  the  campaign-specific  issues  affecting  the  operational 
capabilities  of  those  aircraft  could  assess  a  set  of  combat  aircraft,  unique  to  a  deployment 
plan.  This  research  develops  a  non-absolute,  campaign-specific  decision  support  tool  to 
assess  the  effect  of  campaign-specific  issues  on  a  set  of  deployable  Air  Force  combat 
aircraft. 

This  research  is  conducted  in  three  phases.  Cognitive  Task  Analysis  (CTA)  and 
Multi- Attribute  Preference  Theory  (MAPT)  methodologies  are  applied  to  in-person  and 
telephone  interviews  with  subject  matter  experts  (SMEs)  on  USAF  wartime  planning. 
The  information  is  then  consolidated  to  develop  a  Value  Focused  Thinking  decision 
support  tool.  This  tool  is  developed  with  contingency  planners  at  the  United  States  Ar 
Force’s  Central  Command  Headquarters.  The  decision  support  tool  is  verified  through  a 
Delphi  study  with  the  previously  identified  SMEs. 

The  results  of  this  research  provide  campaign  planners  with  a  decision  support 
tool  to  assist  in  selecting  a  set  of  USAF  combat  aircraft  best  suited  to  deploy  in  response 
to  a  SWA  Theater  crisis,  based  upon  the  current  environment  within  the  SWA  Theater. 
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If  I  always  appear  prepared,  it  is  because  before  entering  an  undertaking, 
I  have  meditated for  long  and  have  foreseen  what  may  occur. 

It  is  not  genius  which  reveals  to  me  suddenly  and  secrecy 
what  I  should  do  in  circumstances  unexpected  by  others; 
it  is  thought  and  preparation. 


Napoleon  Bonaparte 


A  DECISION  SUPPORT  TOOL  TO  AID  CAMPAIGN  PLANNERS 


IN  SELECTING  COMBAT  AIRCRAFT  FOR  THEATER  CRISIS 

I.  INTRODUCTION 


Background 

The  Advanced  Logistics  Project  (ALP),  a  Defense  Advanced  Research  Projects 
Agency  (DARPA)  skunkworks,  is  currently  developing  a  distributed  computing 
architecture  that  will  create  a  near-real-time  deployment  planning  process  for  military 
forces.  This  deployment  development  and  implementation  architecture  (computer 
software  program)  will  enable  logistics  planners  from  the  US  military  services  to  quickly 
and  efficiently  develop  and  implement  a  situation-tailored  logistics  plan.  One  important 
issue  that  the  ALP  has  not  yet  resolved  is  the  ability  of  the  architecture  to  choose  a 
particular  logistics  plan  fi’om  among  a  set  of  similar  alternative  plans.  Consequently, 
ALP  has  requested  the  support  of  both  the  Air  Force  Institute  of  Technology  (AFIT)  and 
the  Air  Force  Research  Laboratory  (AFRL)  to  develop  a  methodology  for  rationally 
measuring  the  relative  contribution  of  each  alternative  deployment  plan  toward  satisfying 
a  given  combat  commander’s  time-phased  campaign  objectives.  Both  AFIT  and  AFRL 
are  taking  a  United  States  Air  Force  (USAF)  focused  view  to  the  research.  If  a 
methodology  can  be  developed  to  rationally  measure  the  relative  contribution  of 
individually  specific  combat  equipment  to  a  particular  deployment  plan,  then  this 
methodology  should  be  expandable  to  incorporate  not  only  USAF  combat  assets  but 
Army,  Navy  and  Marine  Corps  combat  assets. 


One  plausible  method  of  assessing  the  relative  contribution  of  a  unique 
deployment  plan  centers  on  the  specific  set  of  combat  aircraft  tasked  within  each 
alternative  deployment  plan  to  meet  the  combat  commander’s  requirements.  The  purpose 
of  a  USAF  deployment  plan  is  to  provide  the  combat  commander  with  the  combat 
aircraft,  and  necessary  supporting  equipment,  to  meet  the  required  objective.  Therefore, 
a  specific  set  of  combat  aircraft  assets  that  is  best  suited  to  meet  the  needs  of  the  combat 
commander,  versus  other  unique  aircraft  asset  sets,  would  be  one  way  of  identifying  the 
best  deployment  plan.  This,  of  course,  assumes  that  each  of  the  competing  deployment 
plans  are  otherwise  similar  and  that  all  of  the  competing  deployment  plans  could  be 
successfully  implemented. 

When  comparing  competing  sets  of  combat  aircraft  assets,  a  first  concern  is 
ensuring  that  the  aircraft  comprising  the  set  are  capable  of  performing  the  aerospace 
missions  required  by  the  combat  commander.  Each  specific  combat  aircraft  has  been 
designed  and  developed  with  an  absolute  capability  to  perform  one  or  more  specific 
aerospace  missions.  For  example,  the  absolute  capability  of  the  F-15,  C/D  model  is  to 
conduct  Air  Interdiction  missions.  The  absolute  capability  of  an  F-16,  C  model, 
however,  can  conduct  Suppresion  of  Enemy  Air  Defense,  Air  Interdiction,  and  Surface 
Attack  missions,  just  to  name  a  few.  However,  only  looking  at  the  absolute  capabilities 
of  the  combat  aircraft  within  the  asset  set  may  not  satisfactorily  evaluate  comparable 
asset  sets,  as  many  of  today’s  USAF  combat  aircraft  can  be  equipped  to  complete  many 
of  the  possible  aerospace  missions. 

Therefore,  a  second  method  of  comparing  competing  aircraft  asset  sets  is  to  study 
the  situation  dependent  or  contextual  issues  that  may  be  specific  to  the  campaign  or 
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theater  in  which  the  aircraft  asset  set  will  be  deployed.  While  each  individual  combat 
aircraft  asset  may  be  specifically  designed  to  complete  certain  aerospace  missions, 
regional  or  campaign  specific  issues  may  drive  the  necessity  to  complete  aerospace 
missions  with  less-than-optimal  combat  aircraft  assets.  Constraints  placed  upon  US 
forces  by  the  host  nation  or  at  the  staging  base  within  host  nation  territory  may  keep 
campaign  planners  from  selecting  certain  combat  aircraft.  For  example,  host  nation 
political  issues  may  restrict  the  presence  or  use  of  combat  aircraft  perceived  to  be 
‘offensive',  or  nuclear  capable.  Campaign  specific  issues  such  as  these  may  be  a  very 
important  concern  to  the  campaign  planner  in  selecting  the  best  aircraft  to  conduct  the 
required  aerospace  missions.  Above  and  beyond  the  capabilities  of  specific  combat 
aircraft  to  complete  aerospace  missions,  campaign  specific  issues  may  be  an  important 
driver  or  limitation  to  specific  sets  of  combat  aircraft  that  can  be  deployed  to  meet  a 
theater  crisis. 

Problem  Statement 

Given  that  the  Advanced  Logistics  Project  will  provide  a  methodology  to  quickly 
develop  a  detailed  deployment  plan,  campaign  planners  will  have  the  opportunity  to 
compare  multiple  deployment  scenarios  and  schedules.  How  would  campaign  planners 
then  choose  between  similar  deployment  plans?  Different  deployment  plans  may  call  for 
the  selection  of  different  fighter  and/or  bomber  aircraft  assets.  An  assessment  could  be 
based  upon  campaign  specific  or  theater  specific  issues  that  may  not  be  addressed  by  an 
aircraft’s  absolute  capabilities,  but  would  be  nonetheless  important  to  the  satisfactory 
completion  of  the  operational  missions  required  by  that  set  of  assets.  A  particular  set  of 
combat  aircraft,  unique  to  an  individual  deployment  plan,  could  be  assessed  based  upon 
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both  the  absolute  capabilities  of  the  combat  aircraft  within  the  particular  set  and 
contextual,  campaign  specific  issues  that  could  affect  the  mission  capability  of  the 
combat  aircraft  within  the  particular  set.  The  focus  of  this  research,  then,  is  to  assess  the 
contribution  of  contextual  issues  as  these  issues  would  relate  to  and  modify  the  absolute 
capabilities  of  the  aircraft  assets  in  determining  the  relative  merit,  or  value,  of  competing 
sets  of  combat  aircraft.  This  research  will  also  develop  a  decision  support  tool  to  provide 
an  automated  evaluation  of  situation  dependent,  campaign  specific  issues  as  they  relate  to 
a  Southwest  Asian  Theater,  Air  Expeditionary  Force  deployment. 

Research  Questions 

To  successfully  complete  this  research  project,  the  following  questions  must  be 
answered: 

1 .  What  factors  other  than  absolute  aircraft  capabilities  should  campaign  planners 
consider  when  selecting  a  specific  set  of  combat  aircraft  to  counter  a  theater  threat? 

2.  How  are  these  factors  quantified? 

3 .  What  relationships  link  these  factors  to  each  other? 

4.  Based  on  a  Southwest  Asia  scenario,  what  is  the  relative  importance  of  each  factor 
with  respect  to  the  others? 

Methodology 

This  research  is  conducted  in  three  phases.  The  first  two  phases  include  in-person 
and  telephone  interviews,  conducted  with  subject  matter  experts  (SMEs)  on  USAF 
wartime  planning  and  deployments,  using  Cognitive  Decision  Making  (CDM)  and  Value- 
Focused  Thinking  (VFT)  methodologies.  Using  the  information  gleaned  from  these  first 
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two  phases  of  the  research,  a  Value  Focused  Thinking  decision  support  tool  is  developed. 
This  tool  is  developed  with  contingency  planners  at  the  United  States  Air  Force’s  Central 
Command  Headquarters  (CENTAF).  The  third  and  final  phase  of  the  research  verifies 
the  decision  support  tool  through  the  use  of  a  Delphi  study  with  the  previously  identified 
SMEs. 

Assumptions 

This  thesis  research  is  limited  to  a  United  States  Air  Force,  Southwest  Asia 
(SWA)  Theater  Expeditionary  Air  Force  combat  deployment. 

The  environment  in  which  the  military  conducts  its  business  is  constantly 
changing,  primarily  the  result  of  changing  international  politics  and  world  economics. 

The  findings  and  conclusions  of  this  research  are  therefore  based  on  current  international 
politics  and  business  practices,  and  may  not  be  valid  in  a  different  world  order.  While 
U.S.  forces  currently  choose  to  take  the  lead  in  international  military  issues,  a  shift  of  our 
national  forces  to  a  supporting  role  under  United  Nations  (UN)  or  North  American  Treaty 
Organization  (NATO)  lead  campaigns  may  alter  U.S.  planning  processes.  Furthermore, 
the  factors  used  in  the  decision  support  tool  have  been  developed  primarily  through  SME 
opinion  and  expertise  with  respect  to  a  SWA  campaign,  so  the  issues  and  relative 
valuations  obtained  here  may  not  apply  in  another  scenario  or  theater.  Because  of  the 
joint  focus  of  our  forces  under  complementary  missions,  it  is  hoped  that  the 
methodology,  findings,  and  conclusions  from  this  Air  Force  focused  research  can  provide 
a  basis  for  fixrther  research  with  respect  to  campaign  planning  in  other  theaters  and  within 
the  other  branches  of  service. 
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Scope/Limitations 

Because  of  the  need  to  main  secrecy  within  the  planning  of  military  operations, 
portions  of  Concepts  of  Operations  and  Operating  Instructions  may  be  classified.  In 
order  to  keep  this  effort  unclassified,  research  conclusions  will  only  be  considered  within 
the  scope  of  unclassified  information. 

The  focus  of  this  research  is  to  examine  the  issues  or  factors  not  tied  to  the 
absolute  capability  of  USAF  combat  aircraft  to  perform  specific  aerospace  missions  that 
would  affect  the  deployment  suitability  of  particular  aircraft  assets  to  counter  a  theater 
threat.  Because  of  this,  the  absolute  capability  of  each  individual  USAF  combat  aircraft 
is  not  considered  or  studied. 

Furthermore,  campaign  specific  issues,  not  tied  to  the  absolute  capabilities  of 
combat  aircraft  assets,  may  vary  from  theater  to  theater;  therefore,  this  decision  support 
tool  research  focuses  only  on  the  SWA  Theater.  The  Expeditionary  Aerospace  Force 
concept  was  developed  largely  because  of  ongoing  commitments  in  the  SWA  Theater. 
Therefore,  campaign  planners  within  today’s  Concepts  of  Operations  can  most 
effectively  apply  a  decision  support  tool  based  upon  the  SWA  Theater.  However,  the 
campaign  specific  issues  identified  in  this  research  may  be  important  in  other  theaters. 

Summary 

This  chapter  provides  the  motivation  behind  the  development  of  a  United  States 
Air  Force  campaign-specific  decision  support  tool  to  help  campaign  planners  select  the 
best  mix  of  combat  aircraft  to  deploy  in  response  to  a  theater  crisis.  One  way  of 
comparing  alternative  deployment  plans  is  to  examine  how  well  the  specific  set  of 
combat  aircraft  tasked  within  each  deployment  plan  meets  the  combat  commander’s 
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requirements.  A  particular  set  of  combat  aircraft,  unique  to  an  individual  deployment 
plan,  could  be  assessed  based  upon  both  the  absolute  capabilities  of  the  set’s  combat 
aircraft,  and  the  campaign  specific  issues  that  could  affect  the  operational  capabilities  of 
the  combat  aircraft.  This  research  develops  a  non-absolute,  campaign  specific  decision 
support  tool  that  can  be  used  to  assess  the  effect  of  campaign  specific  issues  on  any 
candidate  set  of  deployable  Air  Force  combat  aircraft. 

Chapter  II  reviews  deploying  issues  as  well  as  the  tools  and  techniques  used  to 
elicit  information  from  subject  matter  experts  for  developing  decision  support  tools. 
Chapter  III  describes  the  methodology  used  to  elicit  deployment  factors  and  develop  the 
decision  support  tool.  Chapter  IV  develops  the  decision  support  tool,  and  Chapter  V 
provides  conclusions  and  discusses  the  applicability  of  the  developed  decision  support 
tool  to  both  on-going  AFIT/AFRL  and  ALP  deployment  planning  work  and  Air  Force 
operations  planners. 
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II.  LITERATURE  REVIEW 


INTRODUCTION 

The  primary  mission  of  the  campaign  planner  is  to  provide  the  Joint  Air 
Component  Commander  with  a  satisfactory  set  of  combat  aircraft  to  meet  required 
aerospace  missions.  A  set  of  USAF  combat  aircraft,  or  asset  set,  is  defined  for  the 
purposes  of  this  research  as  any  combination  of  fighter  and/or  bomber  aircraft  that  have 
the  absolute  capabilities  to  satisfy  the  combat  requirements  of  the  Joint  Air  Component 
Commander,  Joint  Forces  Commander,  and  or  Theater  Commander.  When  selecting  a 
set  of  USAF  combat  aircraft  to  deploy,  campaign  planners  must  take  into  consideration 
many  unique  political,  economic,  and  social  considerations,  as  well  as  national  and 
international  goals.  The  decisions  of  these  planners  may  ultimately  decide  the  outcome 
of  the  conflict.  As  this  task  involves  high  stress,  short  time  constraints  and  international 
politics,  experienced  planners  are  essential. 

This  chapter  begins  with  a  presentation  of  the  methodology  behind  Cognitive 
Task  Analysis  and  Cognitive  Decision-Making  methodologies,  and  why  these 
methodologies  were  selected.  Next,  a  discussion  on  Multi-Attribute  Preference  Theory 
and  Value  Focused  Thinking  and  why  these  methodologies  were  selected,  follow  the 
discussion  of  cognitive  methodologies.  This  chapter  continues  with  an  analysis  of  the 
Delphi  study  technique,  which  is  an  iterative  survey  and  reply  method  that  is  used  to 
obtain  feedback  and  agreement  on  the  planning  factor  decision  support  tool.  An 
explanation  of  why  this  methodology  was  selected  is  also  presented.  This  chapter 
concludes  with  an  analysis  of  the  current  operational  status  of  USAF  deployment  theory, 
and  the  current  level  of  detail  on  planning  factors  presented  within  Air  Force  and  Joint 
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Doctrine.  Air  Force  and  Joint  Doctrine  is  used  as  the  foundation  for  understanding  of  the 
planning  factors  that  drive  the  selection  of  combat  aircraft  under  crisis  action  planning. 


IDENTIFYING  THE  BEST  METHODOLOGY  TO  ELICIT  ISSUES  FROM 
EXPERTS 

Several  interviewing  methodologies  were  studied  to  determine  which  method  best 
suited  the  initial  identification  of  campaign-specific  issues.  Knowledge  of  these  issues  is 
limited  to  a  select  group  of  individuals.  Furthermore,  understanding  of  how  the 
campaign-specific  issues  are  interrelated  and  how  these  issues  could  be  qualitatively 
measured  requires  the  aid  of  individuals  with  experience  and  knowledge  of  campaign 
planning.  To  identify  these  campaign-specific  issues  from  campaign  planning  experts, 
three  elicitation  methods  were  identified  for  possible  use.  The  first  method  was  the 
formal  survey.  The  second  method  studied  was  the  Traditional  Task  Analysis.  The  third 
method  identified  for  use  in  this  study  is  the  Cognitive  Task  Analysis  methodology. 

THE  FORMAL  STUDY 

A  formal  Air  Force  survey  is  normally  developed,  approved  by  Air  Force 
Personnel  Command,  and  then  mailed  to  an  identified  group  of  participants.  A  survey 
consists  of  specific  questions,  usually  developed  to  be  answered  using  either  ranges 
(worst  to  best),  multiple  choice  suggestions,  or  fill-in-the-blanks.  Formal  surveys  often 
take  several  weeks  to  be  approved,  then  over  a  week  to  reach  all  of  the  participants,  and 
finally  several  weeks  after  that  to  see  any  responses.  In  addition  to  this  time  constraint, 
most  formal  surveys  receive  responses  from  at  most  one-half  of  the  total  number  sent 
[Steele;  1999].  A  formal  survey  was  not  selected  as  the  method  to  obtain  the  views  and 
experience  from  campaign  planners. 


9 


The  formal  survey  was  discounted  for  three  reasons.  First,  the  purpose  of  the 
study  is  to  obtain  views  and  experience,  which  may  not  be  concisely  captured  in  a  formal, 
static  survey.  The  way  in  which  the  questions  are  asked  may  not  elicit  the  responses 
necessary  to  develop  a  solid  decision  support  tool.  Secondly,  due  to  the  time 
requirements  of  a  formal  survey,  multiple  rounds  of  questions,  if  necessary  to  obtain 
understanding  of  the  issues,  would  not  be  feasible.  Finally,  a  formal  survey  was 
discounted  because  of  the  small  number  of  expert  (rank  of  Major  to  Colonel)  campaign 
planners  actively  conducting  planning  operations. 

TRADITIONAL  TASK  ANALYSIS 

Traditional  job  or  task  analysis  provides  a  deep  understanding  of  individual  jobs, 
their  behavioral  requirements,  and  target  performance,  which  help  to  create  a  better 
understanding  of  how  the  individual  employees  within  an  organization  should  function. 
Traditional  task  analyses  study  job  tasks  in  relation  to  behavior  responses  that  must  be 
made  to  each  stimulus  encountered,  emphasizing  item-specific  knowledge  [Seamster  et 
al.,  1997;  pg  5].  First,  the  specific  job  must  be  defined,  to  include  task  requirements  and 
people  requirements  [Cascio,  1998;  pg  133].  Descriptions  of  the  job  are  useful  only  to 
the  extent  that  they  accurately  represent  job  content,  environment,  and  conditions  of 
employment  [Cascio,  1998;  pg  137].  The  primary  method  of  defining  jobs  and  the 
specific  tasks  of  these  jobs  is  through  direct  observation.  Using  this  method,  the  research 
would  require  the  unobtrusive  observation  of  an  actual  real-world  or  exercise  campaign 
planning  process.  This  method  is  primarily  suited  for  jobs  that  require  a  great  deal  of 
manual,  standardized,  and  short-cycle  activities,  and  inappropriate  for  jobs  that  require  a 
great  deal  of  mental  activity  and  concentration  [Cascio,  1998;  139-140], 


10 


The  traditional  task  analysis  was  discounted  for  three  reasons.  First,  the 
methodology  emphasizes  the  behavior  of  workers  performing  specific  tasks.  The  focus 
of  this  research  is  on  the  identification  of  issues  that  workers,  in  this  case  campaign 
planners,  must  appreciate  when  performing  their  duties.  The  focus  is  therefore 
completely  on  the  task  and  not  on  the  behavior  of  the  worker.  Secondly,  the  task  of 
campaign  planning  is  not  generally  a  rigorously  manual  activity.  Conversely,  the  purpose 
of  this  study  is  to  identify  campaign-specific  issues  that  require  concentration  and  mental 
capability  to  resolve.  Lastly,  there  was  not  any  opportunity  during  the  period  of  this 
research  to  attend  either  a  real-world  or  exercise  campaign-planning  operation. 

COGNITIVE  TASK  ANALYSIS 

Today’s  professions  often  require  a  level  of  knowledge  and  skill  that  may  not  be 
achievable  without  years  of  training  and  practice.  However,  under  many  circumstances  it 
is  not  simply  the  years  of  training  and  practice  that  make  an  individual  qualified.  Often 
times,  organizations  look  for  individuals  who  have  wisdom  and  experience  within  their 
field  that  cannot  be  simply  memorized  from  a  textbook.  Human  thought  and  behavior 
have  a  direct  impact  on  the  capabilities  of  an  individual,  but  may  be  outside  the  normal 
perception  of  job  duties.  For  instance,  an  airport  traffic  controller  may  make  hundreds  of 
critical  decisions  regarding  the  flow  and  order  of  incoming  and  outgoing  aircraft,  all  of 
which  may  go  unnoticed  to  a  casual  bystander  or  a  trainee.  Cognitive  Task  Analysis 
(CTA)  can  be  used  to  study  these  non-observable  skills  and  cognitive  activities.  For 
example,  CTA  methods  can  be  used  to  record  the  air  traffic  controller’s  performance  and 
translate  or  transform  it  through  a  systematic  process  such  as  speeding  up,  slowing  down, 
or  coding  it  so  a  perceptible  pattern  emerges  [Seamster,  1997:  pg  25].  The  resulting 
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information  or  patterns  can  be  used  to  better  understand  these  non-observable  skills  or 
cognitive  actions  of  experts,  allowing  trainees  to  learn  and  improve  their  performance 
more  quickly  than  would  otherwise  be  experienced.  Therefore  CTA  methods  were 
selected  to  perform  the  first  phase  of  this  research. 

Methods  for  CTA  occupy  a  central  position  in  cognitive  science,  whereby 
researchers  work  to  uncover  the  underlying  knowledge  and  thought  processes  that  make 
experts  more  valuable  than  novices.  CTA  can  be  used  to  form  inferences  about  non¬ 
observable  cognitive  activities  through  the  development  of  an  accurate  picture  of  the 
tasks  and  environment  in  which  they  are  performed.  According  to  Klein  et  al.,  [1997:  pg 
1],  task  analysis  methods  usually  provide  a  listing  of  the  steps  to  be  followed,  but  on  top 
of  those  steps  are  the  difficult  and  critical  decisions  that  operators  must  make  under 
restraining  conditions.  We  can  therefore  see  that  decision  requirements  are  not  limited  to 
choices  between  options;  they  can  include  judgments,  situation  assessments,  and 
problem-solving  activities.  CTA  methods  were  selected  to  perform  the  first  phase  of  this 
research.  Because  experience  is  a  key  issue,  CTA  methods  can  be  applied  to  elicit 
information  from  experienced  planners  that  can  then  help  planners  of  all  levels  of 
experience  to  make  better  decisions.  Furthermore,  a  decision  support  tool  can  be 
developed  from  this  cognitive  data. 

In  recent  research  into  the  abilities  of  workers,  Ericsson  and  Charness  [1994:  pg 
725-747]  found  that  expert  performance  results  from  increases  in  knowledge  and  skill 
acquired  through  work  and  practice.  As  one  might  assume,  extended  training  and 
practice  develop  and  refine  memory  processes  and  content  in  such  a  way  that  advances 
expert  performance.  Ericsson  and  Charness’ s  theory  of  skilled  performance  provide  the 
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basis  for  the  allowance  that  CTA  can  be  used  to  form  a  more  complete  understanding  of 
expert  performance. 

A  portion  of  CTA  involves  working  with  contextual  experts  to  determine  how 
they  perform  cognitive  tasks  in  a  superior  fashion,  but  not  all  contacts  need  to  be  actual 
experts.  Analysts  performing  CTA  are  collecting  information  that  can  uncover  strategies 
that  an  operator  might  not  be  able  to  articulate  [Klein  et  al.,  1997;  pg  3].  Useful 
information  can  be  gleaned  from  all  levels  of  expertise.  By  comparing  the  novice  with 
the  expert,  the  unique  characteristics  of  the  expert  become  more  evident  [Seamster,  1997; 
pg  27].  Experts  and  novices  (at  a  particular  task)  use  knowledge  differently.  Recent 
research  has  found  that  novices  focus  on  perceptual  aspects,  whereas  experts  make 
deliberations  on  what  they  see  [Wezel  et  al.,  1996;  pg  359].  Experts  try  to  figure  out  the 
consequences  of  certain  combinations.  Through  the  understanding  of  these  different 
levels  of  knowledge,  expertise,  and  cognitive  reasoning,  practical  training  and 
understanding  can  be  obtained.  Routine  incidents  can  often  be  handled  automatically,  so 
the  participants,  no  matter  the  level  of  expertise,  may  not  be  aware  of  the  types  of 
judgements  they  were  forced  to  make  [Klein  et  al.,  1997;  pg  4]. 

Cognitive  Task  Analyses  are  typically  conducted  in  stages  to  improve 
manageability  and  integration  of  information  about  sub-tasks  within  the  overall  job 
[Seamster,  1997;  pg  43].  Through  stage  analysis,  many  cognitive  components  of  a  large 
job  can  be  successfully  documented  and  understood.  CTA  is  not  a  single  methodology 
applied  to  the  solution  of  a  single  problem.  Rather,  CTA  comprises  a  set  of  analytic 
methods  and  techniques  that  vary  on  a  number  of  dimensions,  including  the  kind  of  data 
that  they  produce.  CTA  includes  methods  to  elicit,  analyze,  and  represent  information 
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about  cognitive  processes  required  for  proficient  task  performance  [Seamster,  1997:  pg 
64],  Researchers  must  select  and  apply  the  technique  most  appropriate  for  their  specific 
project;  often  this  requires  a  combination  or  synthesis  of  several  techniques.  When 
choosing  the  technique  or  combination  of  techniques,  a  number  of  important  factors  must 
be  considered.  These  factors  include  the  time  available  to  the  researcher  to  conduct  the 
study,  the  budget  allotted  to  complete  the  research,  accessibility  to  subject  matter  experts, 
and  data  collection,  analysis,  and  validation  methods.  The  analysis  and  comparison  of 
these  factors  to  the  type  of  cognitive  task  to  be  studied  leads  to  the  selection  of  the 
analysis  method  [Seamster,  1997:  pg  43-61]. 

IDENTIFYING  THE  BEST  CTA  METHODOLOGY  TO  ELICIT  ISSUES  FROM 
EXPERTS 

Three  CTA  methodologies  were  identified  for  possible  use  within  this  research. 
The  first  methodology  is  the  Verbal  Report  method.  The  second  methodology  is  the 
Simplified  Precursor,  Action,  Result,  and  Interpretation  (PARI)  method.  The  third  CTA 
methodology  studied  is  the  Cognitive  Decision-Making  methodology. 

VERBAL  REPORT  METHOD 

The  Verbal  Report  method  asks  the  subject  matter  expert  to  provide  a  running 
commentary  of  the  actions,  re-actions,  and  thoughts  either  while  conducting  the  job  or 
after  the  job  has  been  completed.  These  verbal  descriptions  of  actions  and  thoughts  are 
usually  recorded  for  later  analysis.  Using  these  recordings,  analysts  can  make  inferences 
about  cognitive  processes  based  on  the  knowledge  the  subject  matter  expert  articulates 
during  these  reports.  This  type  of  analysis,  if  conducted  with  a  number  of  experts,  could 
identify  the  campaign-specific  issues  in  question.  Unfortunately,  there  were  not  any 
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opportunities  to  study  campaign  planners  during  a  campaign  planning  exercise,  nor  could 
the  campaign  planners  be  asked  to  spend  time  writing  down  their  last  campaign  planning 
experience.  A  group  of  planners  may  not  have  had  the  same  experience  in  planning 
campaigns;  some  may  have  planned  deployments  for  combat  operations  while  others  may 
have  planned  deployments  for  humanitarian  efforts.  The  variability  of  historical  planning 
and  the  available  time  these  planners  could  be  required  to  spend  documenting  their 
actions  proved  the  Verbal  Report  method  to  be  infeasible  for  this  research. 

SIMPLIFIED  PARI  METHOD 

The  PARI  methodology  is  based  on  a  structured  interview  process  designed  to 
elicit  a  high  level  of  skill  detail.  A  simplified  PARI  methodology  was  adopted  by 
Seamster  et  al.  from  the  full  PARI’s  time-intensive  methodology  for  us  in  an  operational 
environment  [Seamster  et  al.,  1997;  pg  72].  The  simplified  PARI  analysis  is  based  on  a 
structured  interview,  consisting  of  problem  solving  in  an  operational  context.  Rather  than 
having  an  expert  talk  about  performing  a  task  to  the  researcher,  two  experts  talk  to  each 
other  while  the  interviewer  watches  and  listens.  While  one  expert  poses  a  task-specific 
problem,  the  other  expert  solves  the  problem.  Usually  the  researcher  is  one  of  the 
experts.  This  method  is  best  suited  to  tasks  that  have  procedural  tasks,  such  as 
maintenance  operations  or  computer  trouble-shooting.  For  these  jobs,  there  is  usually  a 
specific  order  in  which  individual  tasks  are  carried  out  sequentially. 

The  simplified  PARI  methodology  was  not  used  for  two  reasons.  First,  this 
researcher  does  not  have  the  expertise  or  background  knowledge  to  participate  in  a  PARI 
analysis,  not  is  the  knowledge  present  to  develop  a  campaign  planning  scenario  with 
sufficient  detail  to  identify  the  campaign-specific  issues  sought.  Secondly,  the  PARI 
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methodology  is  best  suited  to  jobs  that  consist  of  procedural  tasks.  Analysis  of 
campaign-specific  issues  may  not  necessarily  be  an  ordered  task;  therefore  the  PARI 
methodology  is  not  the  best  method  to  use  within  this  research. 

CRITICAL  DECISION  METHOD 

The  CTA  method  used  in  this  research  is  the  Critical  Decision  Method  (CDM). 
CDM  is  a  semi-structured  interview  technique,  developed  by  Klein,  Calderwood,  and 
MacGregor  in  1989  through  the  continuation  of  Flanagan’s  critical  incident  technique. 
CDM  is  a  retrospective  interview  strategy  that  applies  a  set  of  cognitive  probes  to  actual 
nonroutine  incidents  that  required  expert  judgment  or  decision-making  [Klein  et  al., 

1989:  pg  464].  The  interviewing  strategy  can  be  used  to  elicit  information  about 
decisions  made  by  experts  in  any  number  of  field  settings.  Decision  making  and  problem 
solving  are  complex  cognitive  tasks  requiring  substantial  skills.  These  tasks  lie  at  the 
heart  of  proficient  performance  and  serve  to  guide  the  overt  motor  behaviors  that  are  the 
meat  of  traditional  task  analyses  [Seamster,  1997:  pg  78].  These  decisions  often  depend 
on  subtle  perceptual  cues  and  assessments  of  rapidly  changing  events  that  could  not  be 
easily  described.  Interviews  and  probes  enable  experts  to  focus  on  and  document  aspects 
of  job  activities  and  performance  that  is  normally  only  tacitly  understood.  This  method  is 
used  successfully  in  a  number  of  studies  and  in  widely  diverse  domains  [Roth,  1992;  pg 
1163].  For  these  reasons,  the  CTA  method  best  suited  to  develop  a  decision  support  tool 
based  upon  the  cognitive  knowledge  of  experts  is  CDM.  The  following  section  discusses 
the  Critical  Decision  Method  as  it  applies  to  obtaining  cognitive  knowledge  through 
interviews  with  subject  matter  experts. 

The  CDM  focuses  on  the  previous  experiences  of  subject  experts  and  applies 
cognitive  probes  to  elicit  the  expert’s  decision  strategies,  perceptions,  operational  cues. 
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and  errors.  The  CDM  is  usually  applied  through  a  ‘storytelling’  interview,  guided  by  the 
interviewer.  Often,  interviewers  can  successfully  incite  the  interviewee  into  detailing  job 
information  through  telling  a  story  about  the  job.  This  method  puts  the  interviewee  at 
ease  about  the  level  of  complexity  of  the  interview,  and  information  is  often  brought  to 
light  that  may  not  have  been  identified  had  the  interviewee  been  simply  asked  to  point 
out  specific  requirements  of  the  job.  Knowing  the  overall  type  of  information  needed  to 
conduct  the  research,  the  interviewer  guides  the  expert  through  a  particular  activity  or 
incident.  The  interviewer  may  need  to  ask  numerous  questions  to  extract  sufficient 
information  to  develop  a  situation.  The  interviewer  will  then  use  this  situation  as  a 
framework  from  which  to  probe  decisions,  judgments,  and  problem  solving.  One  of  the 
most  challenging  and  important  aspects  of  the  interviewer’s  work  is  to  get  the  respondent 
to  answer  the  question  posed.  If  the  interviewee  gives  an  incomplete  answer  or 
misunderstands  the  question,  then  it  is  the  interviewer’s  responsibility  to  get  the 
interviewee  back  on  track  through  careful,  neutral  techniques  [Interview  Research 
Manual,  1976:  pg  15]. 

The  CDM  interview  generally  consists  of  four  sweeps  through  an  incident  or 
activity.  As  might  be  assumed,  the  first  sweep  captures  the  story  of  the  activity.  During 
this  section  of  the  interview,  the  expert  relates,  in  his  or  her  own  words,  a  particular 
incident  that  required  skill  or  expertise.  This  section  helps  the  interviewer  to  understand 
the  dynamics  of  the  incident,  and  to  determine  whether  the  incident  itself  is  suitable  for 
further  examination.  The  second  section  of  the  interview  is  generally  used  to  develop  a 
timeline  for  the  activity.  This  section  helps  to  map  out  the  sequence  of  events  within  the 
activity.  Inconsistencies,  if  found,  can  be  identified  and  discussed.  Through  the 
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discussion  of  activity  events  in  chronological  order,  more  detailed  and  specific 
information  can  be  obtained  from  the  interviewee.  In  the  third  section  of  the  interview, 
the  interviewer  uses  cognitive  probes  to  elicit  more  detailed  information  regarding 
decision  points.  These  cognitive  probes  examine  goals,  cues,  expectancies,  courses  of 
action,  and  required  information.  During  the  fourth  section  of  the  interview,  the 
interviewer  reviews  the  activity  and  documentation  to  discuss  any  errors;  either  errors 
committed  by  the  expert  or  hypothetical  errors  that  might  be  committed  by  people  with 
less  experience  [Seamster,  1997;  pg  185]. 

While  it  may  seem  easy  to  let  the  interviewee  describe  personally  experienced 
incidents,  relying  on  strictly  personal  experiences  can  hamper  the  analysis  of 
performance  on  more  than  one  incident.  It  is  different  to  make  comparisons  among 
multiple  experts  when  the  data  collection  process  probes  a  unique  incident  for  each 
interviewee.  Exposing  subject  matter  experts  to  a  standard  scenario  and  eliciting  their 
decisions  and  actions  within  this  scenario  can  enable  the  analyst  to  study  the  scenario 
itself 

The  issues  of  reliability,  validity,  efficiency,  and  utility  of  CTA  and  the  CDM 
have  been  discussed  at  length  by  Hoffman,  Crandall,  and  Shadbolt.  Through  their 
research,  a  case  study  of  the  CDM  is  pursued,  proving  that  the  method  can  be  effectively 
applied  as  an  elicitation  method  of  cognitive  requirements  for  any  number  of  tasks 
[Hoffman  et  al.,  1998]. 

Once  the  campaign-specific  issues  have  been  identified  through  cognitive 
analysis,  these  issues  should  be  combined  in  such  as  way  that  the  end  product  provides  a 
net  benefit  to  the  campaign  planner.  The  best  methodology  for  assisting  the  campaign- 
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planners  in  making  better  decisions  lies  in  Decision  Analysis  methodology.  The  next  two 
sections  of  this  chapter  discuss  the  field  of  decision  analysis  and  the  methodology  chosen 
for  this  line  of  research. 

IDENTIFYING  THE  METHODOLOGY  TO  CONDUCT  A  QUALITATIVE 
ANALYSIS  OF  THE  ISSUES 

"Although  decision  analysis  (DA)  provides  structure  and  guidance  for  systematic 
thinking  in  difficult  situations,  it  does  not  claim  to  recommend  an  alternative  that  must  be 
blindly  accepted"  [Clemen,  1995;  pg  4].  This  statement  summarizes  the  purpose  of  the 
application  of  decision  analysis  techniques  in  the  identification  and  analysis  of  campaign- 
specific  issues.  For  this  research,  continuing  cognitive  based  analyses  does  not  suffice. 
While  CTA  principles  provided  an  excellent  methodology  to  identify  the  campaign- 
specific  issues  in  question,  CDM  does  not  lend  itself  to  qualitative  decision  support  tools. 
However,  decision  analysis  methodologies  can  be  implemented  to  provide  qualitative 
analysis  of  issues,  whether  deterministic  or  variable. 

Most  decision  analysis  techniques  revolve  around  the  comparison  of  alternatives, 
and  the  tradeoff  between  multiple  competing  objectives.  DA  is  being  used  to  focus  the 
research  on  evaluating  the  possible  issues  that  a  campaign  planner  must  consider  when 
selecting  combat  aircraft  in  response  to  a  theater  crisis,  and  how  these  issues  will  impact 
unique  sets  of  combat  aircraft.  A  competing  objective  for  campaign  planners  could  be 
the  tradeoff  between  sheer  quantity  of  firepower  and  the  amount  of  lift  required  to  deploy 
the  equipment  necessary  to  deliver  the  firepower.  The  most  important  final  result  of  the 
analysis,  however,  is  not  an  exact  answer  to  the  problem,  but  instead  a  ranked  ordering  of 
possible  sets  of  assets  that  can  be  used  to  assist  the  campaign  planner  in  making  better 
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decisions  more  efficiently.  It  is  for  these  reasons  that  Decision  Analysis  was  selected  as 
the  methodology  to  continue  the  analysis  of  the  campaign-specific  issues  into  the 
development  of  a  qualitative  decision  support  tool. 

IDENTIFYING  THE  DECISION  ANALYSIS  METHODOLOGY 

Several  DA  methodologies  were  initially  identified  to  determine  which  method 
best  suited  the  development  of  the  campaign-specific  issues  into  a  decision  support  tool. 
The  first  method  identified  was  the  Advanced  Hierarchy  Process  (AHP).  The  second 
method  studied  was  Value  Focused  Thinking. 

ADVANCED  HIERARCHY  PROCESS 

AHP  was  developed  in  1977  as  a  decision  aid  to  help  solve  unstructured  problems 
in  economics,  social,  and  management  sciences.  The  methodology  enables  decision¬ 
makers  to  structure  a  complex  problem  in  the  form  of  a  simple  hierarchy  and  to  evaluate 
large  number  of  quantitative  and  qualitative  factors  in  a  systematic  manner  under 
conflicting  multiple  criteria  [Cheng  et  al.,  1999;  pg  423].  Use  of  the  AHP  methodology 
involves  breaking  down  a  complex  problem  into  small  constituent  elements  and  then 
structuring  the  elements  into  a  hierarchical  form.  A  series  of  pairwise  comparisons  is 
then  made  between  elements  to  a  ratio  scale.  The  foundation  of  this  methodology  deals 
with  individual  scales  of  judgements;  comparing  several  expert's  views  and  judgements 
made  lead  to  an  inconsistent  decision  support  tool.  As  rankings  of  the  elements  are  based 
on  verbal  values  such  as  "Good"  and  "Very  good",  comparing  elements  is  often 
confusing  if  not  impossible.  The  decision  support  tool,  as  developed,  must  be  easily 
learned  and  applied  by  individuals  who  are  otherwise  naive  to  mathematical 
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methodologies.  For  these  reasons,  AHP  was  discounted  as  the  methodology  for  the 
development  of  qualitative  measures  for  the  campaign-specific  issues. 

VALUE  FOCUSED  THINKING 

The  one  essential  element  of  a  decision  is  the  existence  of  alternatives;  if  you  do 
not  have  alternatives,  you  do  not  have  a  decision  problem  [Kirkwood,  1997;  pg  2].  In 
many  of  today’s  complex  business  and  operating  environments,  decision-makers  are 
faced  with  choosing  between  several  alternatives  to  maximize  one  or  more  objectives. 
Value  Focused  Thinking  [Keeney,  1992:  pg  55]  is  a  very  useful  technique  for  decision 
situations  involving  multiple  and  conflicting  objectives.  Value  Focused  Thinking  (VFT) 
structures  the  decision-maker’s  values  and  develops  a  multi-objective  decision  analysis 
tool  that  can  be  used  to  both  identify  alternatives  that  create  value,  and  to  evaluate 
alternatives.  "With  value-focused  thinking,  you  should  end  up  much  closer  to  getting  all 
of  what  you  want"  [Keeney,  1992;  pg  4]. 

At  the  heart  of  Value  Focused  Thinking  is  the  value  hierarchy.  Values  provide  a 
foundation  of  interest  in  any  decision  situation.  Value  hierarchies  serve  as  graphical 
representations  of  the  important  considerations  that  decision-makers  will  take  into 
account  and  use  to  determine  the  relative  value  of  competing  alternatives. 

Value  hierarchies  are  used  as  guides  for  information  collection,  to  help  identify 
alternatives,  to  facilitate  communications,  and  to  evaluate  alternatives  [Kirkwood,  1997; 
pg  19-23].  There  are  several  accepted  methods  to  developing  hierarchies.  Kirkwood 
identifies  either  a  bottom-up  or  a  top-down  approach  as  acceptable  [Kirkwood,  1997:  pg 
19-23].  In  the  bottom-up  approach,  alternatives  are  known  and  can  be  examined  to 
determine  how  they  differ.  Grouping  the  differences  in  alternatives  together  into  higher 
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and  higher  tiers  forms  higher  level  values  [Kirkwood,  1997;  pg  19-23],  A  similar 
approach,  called  the  Silver  Standard,  identifies  and  names  individual  tasks  that  the 
organization  performs.  The  named  tasks  can  be  grouped  together  in  affinity  diagrams 
and  then  structured  into  different  value  hierarchy  tiers  [Parnell  et  al,  1998;  pg  1340], 

The  top-down  approach  can  be  used  when  alternatives  are  not  well  specified. 
Values  are  built  starting  at  the  highest  tier  and  are  then  broken  into  successively  lower 
tiers.  Typically,  information  for  this  method  comes  from  mission,  vision  or  strategic 
documentation.  This  process  is  called  the  Gold  Standard  in  the  development  of  a  value 
hierarchy  [Parnell  et  al.,  1998;  pg  1338].  The  Gold  Standard  can  be  effectively  applied 
when  the  organization’s  top  decision-makers  can  be  interviewed. 

In  either  approach  described  by  Kirkwood,  values  are  placed  in  the  hierarchy  if 
and  only  if  value  scores  for  a  particular  alternative  would  change  the  ranking  of  that 
alternative  in  respect  to  the  other  alternatives  [Kirkwood,  1997;  pg  19-23]. 

An  evaluation  measure,  often  referred  to  as  an  attribute,  is  used  to  measure  the 
degree  to  which  an  objective  or  a  value  has  been  obtained  [Kirkwood,  1997;  pg  12]. 
Evaluation  measures  provide  quantitative  measures  for  the  decision.  The  evaluation 
measure  converts  a  quantity  from  its  particular  units  to  a  common  set  of  units,  thus 
allowing  the  many  attributes  to  be  combined  into  a  single  measure  of  merit  or  benefit.  A 
value  hierarchy  combined  with  evaluation  measures  and  the  weights  is  a  value  model. 
Generally,  a  value  model  combines  many  measures  into  a  multiple  objective  value 
function  to  measure  an  alternative’s  attainment  of  the  fundamental  objective.  Careful 
development  of  an  evaluation  measure  is  required  if  it  is  to  provide  correct  insight  to  the 
decision-maker. 
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According  to  Keeney,  important  attributes  are  those  that  can  be  measured  and 
embody  appropriate  implicit  value  judgements.  They  measure  what  the  decision-maker 
is  interested  in  and  do  not  measure  other  items.  Ambiguity  can  occur  if  the  attribute 
levels  are  not  well  defined  [Keeney,  1992:  pg  113],  The  clairvoyance  test  can  be  applied 
to  help  determine  whether  a  measure  is  ambiguous  or  not.  If  a  clairvoyant  could  foresee 
the  future  with  no  uncertainty,  that  clairvoyant  should  be  able  to  unambiguously  assign  a 
score  to  the  outcome  from  each  alternative  in  the  decision  problem  [Kirkwood,  1997:  pg 
28].  If  one  person  assigns  a  level  to  an  attribute,  another  person  should  be  able  to 
interpret  the  attribute  level  with  no  loss  of  information  [Keeney,  1992:  pg  116].  If  the 
evaluation  measure  is  ambiguous,  it  will  not  be  understandable. 

Kirkwood  identifies  four  different  types  of  scales  for  measuring  values  in  value 
hierarchies.  A  scale  can  be  either  natural  or  constructed,  and  can  be  either  be  direct  or 
proxy.  Natural  scales  are  those  in  general  use  that  have  a  common  interpretation. 
Constructed  scales  are  developed  specially  for  a  particular  measure.  Direct  scales  are 
used  to  measure  the  quantity  examined.  Proxy  scales  are  correlated  with  the  quantity 
measured,  but  actually  measure  something  different  than  the  value  being  quantified 
[Kirkwood,  1997:  pg  24].  Constructed  attributes  exactly  measure  what  the  objective  is 
meant  to  address.  Proxy  measures  reduce  the  number  of  attributes  needed  for  a  decision 
and  simplify  descriptions  of  the  consequences  [Keeney,  1992:  pg  118-121]. 

Measurement  theory  generally  acknowledges  five  scale  classifications.  These 
scales,  in  order  of  increasing  information,  are  nominal,  ordinal,  interval,  ratio,  and 
absolute.  Table  1  presents  the  scales,  in  increasing  order  of  information,  along  with  their 
definition,  common  allowable  transformation,  and  example. 
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TABLE  1  -  Measurement  Scale  Types  and  Definitions 

SCALE 

TYPE 

DEFINITION 

ALLOWABLE 

TRANSFORMATION 

EXAMPLE 

Nominal 

Completely  arbitrary 

Any  one-to-one 

Team  jersey  numbers 

Ordinal 

Order  preserved 

Monotonically  increasing/decreasing 

Traffic  Quality 

Interval 

Arbitrary  units  and  origin 

Positive  Linear  (a*x  +b) 

Temperature  (°C  or  °F) 

Ratio 

Natural  zero 

Multiplication  by  scalar 

Temperature  (°K) 

Absolute 

No  choice  of  unit  or  zero 

Multiplication  by  identity 

Counting  (l,2,etc.) 

In  decisions  involving  certainty,  three  types  of  functions  are  often  used  to  define 
an  attribute.  These  functions  are  piecewise  linear  functions,  exponential  functions 
[Kirkwood,  1997:  pg  62-68],  and  discrete  functions  [Clemen,  1996;  pg  80],  Kirkwood 
provides  an  exponential  function  and  a  piecewise  linear  function  written  in  Microsoft 
Excel  Visual  Basic  for  converting  raw  scores  to  value  [Kirkwood,  1997;  pg  78-81], 
Kirkwood  also  provides  detailed  implementation  methodology  for  evaluating  value 
models  in  Microsoft  Excel  [Kirkwood,  1997:  pg  75-81],  This  research  methodology 
makes  extensive  use  of  Kirkwood’s  spreadsheet-based  technique. 

The  single  dimension  value  function  converts  a  quantity  being  measured  into 
value.  Value  is  typically  measured  between  0  and  1,  but  can  be  measured  from  either  0 
to  10  or  0  to  100  depending  upon  the  desires  of  the  decision-maker  [Kirkwood,  1997:  pg 
61],  Whichever  scale  used,  it  must  be  used  for  all  measures  within  the  value  model. 

Within  the  context  of  this  research,  a  straw  model  of  deployable  combinations  of 
combat  aircraft  has  been  developed  as  a  part  of  the  Air  Force  Institute  of  Techno  logy/ Air 
Force  Research  Laboratory  research  [Swartz,  1999],  Within  this  straw  model,  two  basic 
aircraft  have  been  identified:  a  bomber,  with  two  potential  configurations;  B-A  and  B-B; 
and  a  fighter,  with  three  potential  configurations;  F-A,  F-B,  and  F-C.  Certain 
configurations  of  the  different  aircraft  can  provide  higher  or  lower  levels  of  absolute 
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capability  with  respect  to  completing  aerospace  missions  required  by  the  theater 
commander.  Using  these  combat  aircraft  in  their  possible  configurations,  unique  asset 
sets,  containing  different  quantities  of  the  combat  aircraft  in  their  possible  configurations 
can  be  developed.  These  unique  asset  sets  can  then  be  analyzed,  based  upon  their 
absolute  capabilities  and  their  campaign  specific  goodness,  to  determine  which  asset  set 
best  satisfies  the  theater  commander’s  requirements. 

The  purpose  of  the  multiple  objective  value  function  within  this  research  is  to 
select  the  best  available  alternative  based  upon  the  values  identified.  Scoring  alternatives 
is  straightforward,  but  can  be  time  consuming.  Each  unique  deployable  combat  aircraft 
set  will  have  a  score  that  falls  within  the  assigned  range  of  each  value  function.  For  each 
evaluation  measure,  each  set  of  combat  aircraft  will  be  given  a  value  from  0  to  1 
depending  upon  the  single  dimension  value  function.  Some  measures  within  the  value 
hierarchy  will  actually  be  constraints.  For  these  measures,  the  combat  aircraft  set  will 
either  receive  a  multiple  objective  score  of  1,  denoting  it  passes  the  constraint,  or  a 
multiple  objective  score  of  0,  denoting  it  does  not  pass  the  constraint. 

The  method  of  swing  weights  is  commonly  used  to  assess  the  weights  for  the 
values  in  a  hierarchy,  although  other  methods  are  available  and  often  applied  [Clemens 
1996:  pg  546-552:  Kirkwood,  1997:  pg  68-70].  The  swing  weight  method  is  a  thought 
experiment  whereby  the  decision-maker  can  directly  compare  individual  attributes  by 
imagining  (typically)  hypothetical  outcomes  [Clemens  1996:  pg  547].  Each  tier  of  values 
or  measures  in  the  value  hierarchy  is  considered  individually.  It  is  very  important  to  keep 
in  mind  the  range  of  scores  for  each  particular  measure;  a  change  in  the  range  of  a 
particular  measure  can  change  the  rankings  of  each  alternative  [Kirkwood,  1997:  pg  58- 
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59],  To  determine  weights,  the  decision-maker  or  the  decision-maker’s  representative  is 
asked  to  rank  the  measures  from  least  important  to  most  important.  The  ranges  for  each 
measure  are  then  studied  singularly.  The  measures  will  be  studied  by  “swinging”  the 
value  of  the  measure  from  its  lowest  to  its  highest  value.  The  measures  can  then  be 
compared.  The  decision-maker  or  representative  will  be  asked  to  define  the  amount  of 
“swing”  (from  lowest  to  highest  value)  that  measure  “A”  would  need  to  equal  the 
“swing”  of  measure  “B”  from  its  lowest  to  highest  value.  Given  this  question,  the 
decision-maker  or  representative  should  give  a  value  of  importance,  or  weight,  between  0 
and  1  for  measure  “A”  compared  to  measure  “B”  as  shown  in  (1), 

0.75A=B  (1) 

In  this  example,  the  entire  range  of  measure  “B”  is  equal  to  75  percent  of  the  entire  range 
of  measure  “A.”  This  process  continues  until  there  is  one  less  equation  than  the  total 
number  of  categories.  The  final  equation  required  is  the  summation  of  all  measures  to  1 . 
At  this  point,  there  is  a  solvable  system  of  equations  with  an  equal  number  of  equations 
and  unknown  measures. 

After  alternatives  are  scored  and  weights  are  assessed  for  the  values,  all  values  are 
combined  into  a  single  multiple  objective  value  equation.  The  most  common  method  of 
accomplishing  this  is  to  use  an  additive  value  function.  The  additive  value  multi¬ 
objective  function  is  defined  as: 

=  (2) 

i=\ 

Where  x  is  the  overall  evaluation  objective,  x,,  is  the  raw  score  of  attribute  /,  v,  is  the 
single  dimension  value  function,  w,  is  the  weight  of  importance  placed  on  attribute  i,  and 
n  is  the  total  number  of  evaluation  measures.  Keeney,  Kirkwood,  and  Clemen  all  discuss 
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the  necessary  and  sufficient  conditions  for  using  an  additive  value  function  with  value 
hierarchies  [Keeney,  1997:  pg  132-138:  Kirkwood,  1997:  pg  238-239:  Clemen,  1996:  pg 
579-580],  Mutual  preferential  independence  of  attributes  or  evaluation  measures  is 
required  before  using  an  additive  value  function  under  conditions  of  certainty.  Both 
Clemen  and  Kirkwood  provide  formal  testing  procedures  for  mutual  preferential 
independence  [Clemen,  1996:  pg  580-582:  Kirkwood,  1997:  pg  238-239], 

Kirkwood  recommends  that  the  results  be  rank  ordered  after  the  values  (in  this 
research  for  sets  of  deployable  combat  aircraft)  have  been  identified.  Among  other  tools, 
a  graphical  representation  of  each  value’s  contribution  in  the  hierarchy  is  a  useful  tool  in 
presenting  the  importance  of  individual  measures  [Kirkwood,  1996:  pg  76-81], 

Sensitivity  analysis  on  the  weights  can  be  performed  to  determine  the  weighting 
levels  that  will  change  the  alternative  rankings  and  can  determine  what  the  changes  in 
policy  would  be.  If  alternatives  are  insensitive  to  meaningful  variation  in  weights,  further 
research  is  unnecessary.  If  measure  weightings  are  sensitive,  analysis  of  the  weights  can 
be  focused  on  specific  alternatives  or  groups  of  alternatives  [Kirkwood,  1997:  pg  82-85], 
Sensitivity  analysis  is  performed  by  changing  the  weight  of  a  single  evaluation  measure, 
while  holding  all  other  weights  to  the  same  ratio  as  the  developed  case.  All  weights  for 
an  evaluation  measure  tier  must  sum  to  1 .  Therefore,  weights  can  often  times  be  varied 
from  0  to  1.  Kirkwood  provides  specific  guidance  on  the  implementation  of  sensitivity 
analysis  in  a  Microsoft  Excel  spreadsheet  [Kirkwood,  1997:  pg  82-85], 

No  decision  support  tool  is  completed  or  useful  to  a  decision-maker  if  it  has  not 
first  received  approval  from  the  decision-makers  themselves.  Without  buy-in  from  the 
decision-maker,  a  decision  support  tool  is  useless  at  best.  Across  a  spectrum  of  campaign 
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planning  experts,  a  consensus  on  the  viability  of  the  decision  support  tool  would  best 
serve  to  validate  the  model.  The  following  section  describes  possible  methodologies,  and 
the  method  selected  to  obtain  constructive  feedback  and  a  consensus  on  the  developed 
decision  support  tool. 

IDENTIFYING  THE  METHODOLOGY  TO  OBTAIN  CONSENSUS  OF  THE 
DECISION  SUPPORT  TOOL 

Several  methodologies  were  initially  identified  to  obtain  a  consensus  of  the 
decision  support  tool  from  campaign  planners.  The  first  method  considered  was  the 
formal  survey,  to  be  sent  to  campaign  planning  experts.  However,  the  same 
considerations  that  discounted  this  methodology  from  being  used  in  the  identification  of 
the  issues  kept  this  methodology  from  being  used  to  obtain  consensus  of  the  decision 
support  tool.  The  second  method  considered  involved  structured  telephone  interviews, 
coupled  with  e-mail  documents  sent  to  the  experts  to  provide  a  visual  medium  with  which 
to  conduct  the  interview.  This  method  would  take  considerable  time  to  administer,  and 
would  not  allow  a  compilation  of  thought  between  the  subject  matter  experts  polled. 

Also,  documentation  of  the  telephone  interviews  would  be  difficult  to  maintain  direct 
quotation  of  the  expert  interviewed.  The  third  methodology  identified  is  the  Delphi  study 
methodology.  For  the  reasons  described  in  the  following  section,  the  Delphi  study  was 
implemented  to  obtain  consensus  of  the  developed  decision  support  tool. 

DELPHI  STUDY  TECHNIQUES 

The  basic  notion,  theoretical  assumptions,  and  methodological  procedures  of  the 
Delphi  technique  originated  in  the  1950s  and  1960s  by  Dr.  Olaf  Helmer  and  Norman 
Dalkey  at  the  RAND  Corporation  as  a  method  to  obtain  long-range  forecasts.  Since  then. 
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the  technique  has  been  used  in  numerous  situations  that  require  creative  solutions  to  a 
problem  [Oxenfeldt,  1978:  pg  167],  The  technique  is  based  on  a  structured  process  for 
collecting  and  distilling  knowledge  from  a  group  of  experts  by  means  of  a  series  of 
questionnaires  interspersed  with  controlled  opinion  feedback  [Adler,  1996:  pg  3], 
According  to  Dalkey,  the  rationale  and  use  of  the  Delphi  Method  represents  a  systematic 
effort  to  make  the  most  of  what  is,  admittedly,  an  unsatisfactory  situation,  and  to  try  and 
obtain  the  relevant  intuitive  insights  of  experts  and  use  their  informed  judgement  as 
systematically  as  possible  [Adler,  1996:  pg  4], 

The  Delphi  technique  uses  controlled  feedback  to  reduce  the  amount  of 
extraneous  material  or  noise  that  would  otherwise  inhibit  the  process  [Breaux,  1997:  pg 
17],  Dalkey  defines  noise  as  "irrelevant  or  redundant  material  that  obscures  the  directly 
relevant  material  offered  by  participants"  [Dalkey,  1967:  pg  3],  As  part  of  this  feedback 
process,  the  researcher  filters  out  irrelevant  data  and  summarizes  any  and  all  pertinent 
information  obtained  from  the  participants.  The  summarized  information  is  then  returned 
to  the  elicitation  group  in  successive  iterations  in  an  effort  to  arrive  at  a  final  solution  or 
conclusion  [Dalkey,  1968:  pg  3-4],  Each  group  member's  justification  and  reasoning  for 
the  answer  provided  is  included  in  the  summarization  (with  anonymity  upheld).  This  is 
provided  "to  simulate  the  experts  into  taking  into  due  account  considerations  they  might 
through  inadvertence  have  neglected,  and  to  give  due  weight  to  factors  they  were  inclined 
to  dismiss  as  unimportant  on  first  thought"  [Brown,  1968:  pg  3,6]. 

The  Delphi  technique  has  several  advantages  over  other  information  elicitation 
techniques.  A  key  advantage  is  anonymity.  In  a  typical  group  environment,  a  single, 
dominant  individual  can  oftentimes  sway  individual  opinions.  Subordinates  may  also  not 
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want  to  cause  problems  with  their  superiors  by  bringing  up  potential  problems.  The 
Delphi  technique  eliminates  the  effects  of  a  dominant  individual  because  the  responses 
from  all  participants  are  anonymous  [Brown,  1968:  pg  2-3],  Another  advantage  of  the 
Delphi  technique  is  convenience.  Experts  within  a  specific  field  of  study  can  be 
contacted  and  included  at  virtually  any  location  in  the  world,  as  long  as  they  can  receive 
and  send  communications  [Oxenfeldt,  1978:  pg  167].  A  third  advantage  offered  is  the 
minimization  of  pressure  to  conform  to  the  group.  A  statistical  group  response  is 
included  as  part  of  the  summaries  provided  to  the  group  participants.  However,  since 
minority  views  can  be  included  in  the  final  result,  there  is  no  group  pressure  to 
compromise  [Brown,  1968:  pg  6:  Dalkey,  1967:  pg  3-4].  This  removal  of  group  pressure 
eliminates  a  major  bottleneck  in  most  group  dynamics  by  providing  opportunities  for  a 
clear  delineation  of  differing  views  in  a  non-threatening  environment  [Adler,  1996:  pg  7]. 

In  almost  every  application  of  the  Delphi  Method,  two  distinct  phases  can  be 
identified.  The  first  is  defined  as  the  exploration  phase,  whereby  one  or  two  series  of 
questionnaires  are  used  to  fully  explore  and  elicit  additional  information  regarding  the 
subject  in  question.  The  key  to  this  phase  is  ensuring  all  members  of  the  expert  group 
understand  the  aim  of  the  Delphi  study,  understand  the  terminology  and  methods  used, 
and  are  actually  competent  and  confident  in  the  material  under  study.  Expert  group 
members  do  not  need  to  be  experts  in  Delphi  techniques  to  accomplish  the  tasks  required. 

The  second  phase,  or  evaluation  phase,  usually  involves  the  process  of  assessing 
and  gathering  the  expert  group’s  views  on  various  ways  of  addressing  the  issues  of  the 
study.  It  is  important  to  note  that  the  views  obtained  from  the  expert  group  may  not 
necessarily  form  a  consensus;  disagreements  and  differing  views  on  the  subject  in 
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questions  may  be  just  as  useful  to  a  better  understanding  of  the  issues  surrounding  the 
study.  The  evaluation  phase  is  generally  conducted  within  one  or  two  series  of 
questionnaires  following  the  evaluation  phase.  Group  experts  are  usually  asked  to  rank 
items  and  to  establish  preliminary  priorities  among  items  of  discussion.  This  can  help  to 
prioritize  items  of  agreement  and  disagreement  among  group  experts  as  well  as  identify 
issues  requiring  further  explanation. 

The  determination  of  appropriate  experts  and  group  size  are  also  important 
preliminary  requirements  to  the  Delphi  technique.  Experiments  carried  out  in  the  1950s 
and  1960s  show  that  there  is  an  improvement  in  the  quality  of  the  group  outcome  with 
increasing  group  size,  to  a  certain  threshold  [Adler,  1996:  pg  14].  Above  this  threshold, 
including  additional  experts  provides  only  a  marginal  benefit  to  the  distillation  process. 

Expertise  is  usually  a  key  requirement  in  selecting  members  for  a  Delphi  panel. 
The  purpose  of  using  experts  specific  to  the  aim  of  the  study  is  to  elicit  responses  and 
solutions  to  the  study  that  are  more  meaningful  than  if  just  anyone  participates  in  the 
study.  Selection  of  these  experts  is  generally  not  conducted  as  a  matter  of  personal 
preference;  rather,  it  must  follow  explicit  criteria  depending  upon  the  aims  and  context 
within  which  the  Delphi  process  is  to  be  carried  out  [Alder,  1996;  pg  14].  Potential  group 
members  must  have  knowledge  and  practical  engagement  with  the  issues  under 
investigation.  Potential  group  members  must  also  have  the  capacity  and  willingness  to 
contribute  to  the  study  conducted.  As  with  any  writing  intensive  activity,  potential  group 
members  must  be  able  to  clearly  express  themselves  through  written  communication. 
Academic  qualifications  should  not  preclude  a  potential  group  member  from  the  study, 
unless  the  aim  of  the  study  specifically  requires  such  requirements. 
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To  increase  the  chances  that  a  study  group  will  be  creative  and  synergistic  in 
thinking,  the  group  should  be  united  in  their  effort  to  accomplish  their  task.  One  simple 
method  to  help  accomplish  this  synergy  and  creativeness  is  to  use  a  straw  model.  A 
straw  model  is  a  conceptual  model  of  the  group’s  task.  It  defines  the  parameters  of  the 
task  and  presents  a  perspective  on  how  the  task  can  be  accomplished  [Adler,  1996:  pg 
43].  This  model  need  not  be  complete,  and  it  may  include  inaccuracies  or  omissions  in 
information.  The  straw  model  helps  to  make  a  group  aware  of  the  areas  where  they  must 
concentrate  their  efforts  and  to  provide  a  target  for  a  group  to  accomplish. 

Why  do  campaign  planners  need  a  decision  support  tool  to  assist  them  in 
identifying  and  managing  campaign-specific  issues?  This  section  discusses  the  current 
state  of  Air  Force  operations  as  implemented  through  the  Air  Expeditionary  Force 
concept.  The  current  concept  of  operations  shows  the  need  for  campaign  planners  to 
obtain  a  decision  support  tool  that  will  improve  the  efficiency  and  effectiveness  of 
campaign  planning,  leading  to  the  deployment  of  the  right  assets  to  meet  the  mission. 

CURRENT  STATE  OF  AIR  FORCE  OPERATIONS 

"Global  Engagement:  A  Vision  for  the  21st  Century  Air  Force"  is  the  result  of  an 
18-month  long-range  planning  effort  [Global  Engagement  WWW  paper].  Since 
November  of  1996,  Global  Engagement  has  succeeded  "Global  Reach-Global  Power"  as 
our  air  force's  strategic  direction  into  the  next  century,  providing  the  vision  behind  how 
our  air  forces  will  fight. 
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During  the  Cold  War,  the  well-manned  stateside  Air  Force  concentrated  on 
generating  and  launching  aircraft  from  the  continental  United  States  (CONUS)  to 
reinforce  the  significant  number  of  forces  well  established  in  the  European  or  Pacific 
theaters.  As  a  result  of  the  end  of  the  Cold  War,  the  need  for  significant  forward 
presence  to  deter  Communist  forces  diminished.  As  seen  during  ever  other  time  of 
relative  peace,  the  United  States  made  dramatic  reductions  in  its  number  of  armed  forces. 
Most  of  these  forces  came  from  overseas  locations,  significantly  drawing  back  our  global 
presence.  However,  as  our  nation's  forces  were  drawing  down  and  back,  Iraqi  aggression 
made  it  clear  that  we  still  must  be  able  to  strike  quickly  and  with  decisive  force  to  any 
location.  According  to  Global  Engagement,  CONUS-based  forces  will  become  the 
primary  means  for  this  expedient  power  projection  and  crisis  response. 

The  new  vision  of  our  Air  Force  is  built  upon  six  core-competencies:  air  and 
space  superiority,  global  attack,  rapid  global  mobility,  precision  engagement,  information 
superiority,  and  agile  combat  support.  Air  and  space  superiority  not  only  brings  our 
forces  freedom  from  attack,  but  allows  us  to  dominate  the  other  person's  airspace.  This 
dominance  provides  further  freedom  to  attack  targets  across  enemy  territory  while 
denying  sanctuary  to  enemy  forces.  If  we  have  air  and  space  superiority,  our  forces  will 
then  naturally  want  to  be  able  to  precisely  engage  the  enemy.  Precision  engagement  goes 
beyond  precision  weapons  to  include  precision  airlift,  aerial  resupply,  and  the  precision 
that  comes  from  decisively  attacking  an  adversary's  command  and  control  system  at  the 
right  place  and  at  the  right  time.  Global  attack,  rapid  global  mobility,  and  agile  combat 
support  go  hand-in-hand.  The  Air  Force  must  be  able  to  hit  targets  while  operating  from 
CONUS  bases,  but  must  also  be  able  to  provide  expeditious  deployment  and  sustainment 
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around  the  world.  Information  superiority  entails  being  able  to  both  defend  our 
information  systems  and  attack  the  enemy's. 

AEF/AEG  CONCEPT 

The  key  to  US  global  dominance  is  the  ability  to  project  power  quickly  any  polace 
in  the  world.  The  Air  Expeditionary  Force  (AEF)  and  Air  Expeditionary  Group  (AEG) 
concepts  have  become  the  primary  Air  Force  tools  for  realizing  the  Global  Engagement 
vision  of  CONUS-based  crisis  response  and  action.  According  the  Secretary  of  Defense 
William  S.  Cohen,  the  primary  AEF  mission  is  to  provide  regional  commanders  in  chief 
(CINCs)  and  Joint  force  commanders  with  vital  air  and  space  forces.  These  forces  can 
carry  out  wide  ranging  airpower  options,  meeting  specific  theater  needs  across  the  full 
spectrum  of  military  response  options  [Cohen,  1998].  The  AEF/AEG  concept  fits  nicely 
within  the  core  competencies  of  the  Air  Force.  According  to  Brigadier  General  William 
R.  Looney  III,  the  AEF  makes  the  final  transition  from  a  force  founded  on  the  strategy  of 
forward-based  presence  to  one  built  on  the  vision  of  global  engagement  [Looney,  1996: 

5].  An  AEF  is  an  airpower  package  that  usually  consists  of  30  to  40  aircraft.  This 
package  is  developed  to  provide  theater  commanders  with  rapid,  responsive,  and  reliable 
airpower  capabilities  and  options  that  meet  specific  theater  needs  [Looney,  1996;  6].  The 
goal  of  the  AEF  is  to  launch  combat  sorties  in-theater  48  hours  after  an  'execute  order'  is 
issued  and  then  sustain  combat  airpower  for  the  duration  of  the  conflict  or  crisis.  A 
typical  AEF  package  comprises  30  aircraft;  12  air  superiority,  12  strike,  and  six 
Suppression  of  Enemy  Air  Defense  (SEAD)  fighter  aircraft  [Looney,  1996;  6].  This 
package  was  later  increased  to  36  aircraft  by  adding  an  additional  six  SEAD  fighter 
aircraft  [Godfrey,  1998;  1].  However,  due  to  the  variable  and  unforeseeable  nature  of 
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enemy  actions  and  theater  requirements,  AEF  packages  can  be  tailored  to  meet  specific 
needs  and  threats.  Tanker,  Cargo,  and  Bomber  assets  can  be  added  as  needed  or  required. 
Bomber  assets  can  be  provided  from  either  CONUS  based  locations,  or  deployed  to  a 
forward  location,  near  the  threat,  that  is  capable  of  taking  these  aircraft. 

The  Chief  of  Staff  of  the  Air  Force,  General  Michael  Ryan,  set  a  target  of  1  Oct 
99  to  initiate  the  AEF  concept  across  the  Air  Force.  Currently,  two  of  the  10  AEF  force 
packages  sit  on  what  is  similar  to  alert  status,  rotating  every  90  days.  These  two  AEFs 
will  then  stand-down  to  a  training  and  recovery  status  for  a  period  of  12  months,  while 
other  AEFs  stand  alert.  The  two  AEFs  will  then  stand  back  up  on  alert,  completing  the 
1 5-month  cycle. 

If  an  AEF  is  called  to  deploy,  the  tasked  units  plan  for  a  seven-day  operation 
[Katzaman,  1998:  1].  During  this  operation,  the  AEF  will  provide  a  quick,  sustained, 
initial  strike  capability,  designed  to  halt  (or  at  least  delay)  the  advance  of  an  enemy 
[Godfrey,  1998:  2].  As  the  AEF  is  engaging  the  threat,  CONUS  based  forces  are 
preparing  to  complement  or  reinforce  the  AEF  forces. 

The  question  of  the  proper  mix  of  required  fighter  and  bomber  aircraft  is  of 
critical  importance  to  the  theater  commander.  Under  the  example  of  a  typical  AEF 
package,  the  threat  may  be  able  to  be  satisfied  with  an  enhanced  squadron  of  F-16s. 
Another  threat  however  may  require  a  significant  presence  of  bomber  aircraft.  In 
determining  which  aircraft  should  be  called  upon  to  meet  the  threat,  the  aircraft  selected 
must  be  able  to  meet  the  required  missions.  Thus,  it  is  the  campaign  planner's 
responsibility  to  meet  the  needs  of  the  theater  commander  with  the  best  possible  set  of 
combat  aircraft  for  the  least  cost  possible.  The  selection  of  aircraft  must  be  based  on  not 
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only  the  absolute  capabilities  of  the  aircraft,  but  the  campaign-specific  considerations  that 
may  limit  the  capabilities  of  those  aircraft. 

The  next  section  discusses  current  campaign  planning  operations.  This  research 
focuses  on  developing  a  decision  support  tool  to  aid  campaign  planners  in  selecting  the 
best  value  set  of  combat  aircraft  to  deploy  in  response  to  a  theater  crisis.  To  develop  a 
tool  that  improves  current  campaign  planning  operations,  the  current  state  of  campaign 
planning  must  be  discussed. 
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PLANNING  FOR  WAR 


At  both  national  and  departmental  levels,  various  processes  and  systems  have 
been  developed  to  handle  the  complex  problems  of  setting  strategic  direction, 
determining  national  military  policy,  requesting  resources  to  execute  that  policy,  and 
translating  the  funded  military  capability  into  military  operations.  The  joint  planning 
process  is  one  link  in  a  long  and  complex  chain.  The  purpose  of  joint  operation  planning 
is  to  use  the  military  element  of  national  power  effectively  to  protect  U.S.  interests.  Joint 
planning  is  a  process,  a  systematic  series  of  actions  or  procedures,  used  by  a  commander 
to  determine  the  best  method  of  accomplishing  assigned  tasks  [JSOG,  1997:  para  500]. 
The  Joint  Operation  Planning  and  Execution  System  (JOPES)  is  the  integrated, 
conventional  command  and  control  system  designed  to  satisfy  the  information  needs  of 
senior  decision-makers  in  conducting  joint  planning  and  operations.  The  Joint  Planning 
and  Execution  Community  (JPEG)  use  JOPES  to  conduct  joint  planning  during  peace  and 
crisis.  Joint  operation  planning  must  be  coordinated  through  all  levels  of  the  national 
structure  for  joint  planning  and  execution,  including  the  National  Command  Authority 
and  the  JPEG.  The  focus  of  JOPES  is  centered  on  the  combatant  commanders,  who  use  it 
to  determine  the  best  method  of  accomplishing  desired  or  assigned  tasks  and  direct  the 
actions  necessary  to  accomplish  the  mission.  During  Crisis  Action  Planning  (CAP), 
JOPES  facilitates  the  rapid  development  of  effective  options  and  operation  orders 
through  the  adaptation  of  approved  operation  plans.  JOPES  is  also  an  effective 
management  tool  for  the  execution  of  operations  across  the  spectrum  of  mobilization, 
deployment,  employment,  sustainment,  and  redeployment  [JSOG,  1997:  para  506]. 
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JOPES  implements  policies,  procedures,  personnel,  and  facilities  by  interfacing 
with  automated  data  processing  systems  currently  available  on  the  Global  Command  and 
Control  System.  JOPES  supports  senior-level  decision-making  through  the  staffs  at  the 
NCA  level  and  throughout  the  JPEC. 

JOPES  planning  is  based  on  both  requirements  and  capabilities.  Military  planners 
use  forces  and  resources  identified  in  orders  from  the  Joint  Chiefs  of  Staff,  Service 
documents,  and  approved  operations  plans  and  orders.  JOPES  identifies  the  required 
level  offerees  and  resources  to  accomplish  the  mission,  and  then  compares  them  to 
actual  forces  and  resources  available  based  upon  current  availability  and  other 
commitments.  If  adequate  forces  and  resources  cannot  be  provided  to  the  required  level, 
planners  conduct  a  risk  analysis,  adjusting  the  Commander-in-Chief  s  (CINC’s)  concept 
of  operations  to  an  acceptable  level  of  risk. 

Four  Chairman  of  the  Joint  Chiefs  of  Staff  Manuals  (CJCSM)  describe  the 
operation  and  use  of  JOPES.  Two  of  these  manuals  provide  information  relevant  to  this 
research.  CJCSM  3122.01  (Joint  Pub  5-03.1),  Joint  Operation  Planning  and  Execution 
System  Volume  I  (Planning  Policies  and  Procedures),  unclassified,  describes  the  policies 
and  procedures  governing  the  joint  conventional  deliberate  and  crisis  action  planning 
processes  under  JOPES.  CJCSM  3 122.03,  Joint  Operation  Planning  and  Execution 
System  Volume  II  (Planning  Formats  and  Guidance),  unclassified,  examines  operation 
plan  formats  and  gives  guidance  for  joint  conventional  planning  and  execution  under 
JOPES.  Joint  Publication  3-56. 1,  Command  and  Control  of  Joint  Operations, 
unclassified,  and  Joint  Publication  5-00,  Doctrine  for  Planning  Joint  Operations, 
unclassified,  provide  crisis  action  planning  guidance  fi'om  a  joint  perspective. 
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The  functional  stmcturing  of  joint  operation  plans  in  JOPES  annexes  and 
appendices  is  not  specifically  directed  to  the  unique  organizational  structure  of  each  of 
the  military  services.  Because  of  this,  the  Air  Force  requires  additional  functional 
annexes  to  those  prescribed  in  JOPES.  Air  Force  Manual  10-401  Volume  I,  Operation 
Plan  and  Concept  Plan  Development  and  Implementation,  unclassified,  and  Air  Force 
Manual  10-401  Volume  II,  Planning  Formats  and  Guidance,  unclassified,  provides 
JOPES  focused  planning  information  from  a  more  specific  USAF  perspective.  It  is 
interesting  to  note  that  USAF  planning  guidance,  while  ‘air-centric,’  does  focus  on  the 
need  to  strategically  plan  from  a  joint  perspective.  Air  Force  Doctrine  Document  2, 
Organization  and  Employment  of  Aerospace  Power,  as  well  as  information  from  Air 
University’s  Joint  Doctrine  Air  Campaign  and  Contingency  Wartime  Planning  Courses, 
provide  additional  information  on  planning  requirements  and  issues  specific  to  the  Air 
Force. 

Joint  Publication  5-03. 1,  Joint  Operation  Planning  and  Execution  System  Vol  I 
(Planning  Policies  and  Procedures),  provides  information  that  commanders  should 
address  when  planning  the  deployment  of  forces  in  response  to  a  crisis  situation.  Crisis 
situations  are  generally  classified  as  small,  less  then  major-theater- war  scale  operation 
that  may  require  force  as  all  other  political  and  diplomatic  options  are  failing.  A  recent 
example  of  a  crisis  situation  was  Kosovo  in  the  Spring  1999,  where  combat  aircraft  were 
required  to  deploy,  conduct,  and  sustain  combat  operations. 

The  review  of  joint  publication  5-03.1,  particularly  Chapters  3,  5,  and  Annexes  C, 
D,  and  P,  was  useful  in  the  development  of  the  initial  value  hierarchy  by  this  researcher. 
In  Annex  C,  the  Chairman  of  the  Joint  Chiefs  of  Staff  sends  a  warning  order  to 
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operational  forces  that  may  be  required  to  deploy.  The  warning  order  will  provide 
information  on  the  type  of  missions  required,  political  considerations  present  in  and 
around  the  enemy  location,  enemy  capabilities  and  current  operation,  and  host-nation 
considerations.  Annex  D,  the  Commander ’s  Estimate,  provides  the  theater  commander 
with  the  opportunity  to  add  additional  information  that  may  or  may  not  be  present  in  the 
warning  order.  Annex  P  provides  further  information  on  the  development  of  the 
Commander ’s  Estimate.  This  information  may  include  the  required  operational 
capabilities  needed  to  engage  the  crisis,  staging  base  considerations,  and  more  detailed 
enemy  capability  information.  The  theater  commander  normally  will  provide  his  staff, 
subordinate  commanders,  and  supporting  commanders  with  pertinent  initial  planning 
guidance  to  permit  work  to  begin  on  developing  the  theater  commander’ s  strategic 
concept  [JPUB  5,03-1,  1993:  pg  269].  The  staffs  use  this  guidance  to  begin  developing 
the  Staff  Estimates  that  will  be  used  to  form  the  Commander ’s  Estimate.  Typical  data 
provided  in  preliminary  guidance  such  as  the  Commander 's  Estimate  includes 
characteristics  of  the  area  of  operations,  enemy  capabilities,  the  mission  statement, 
assumptions,  special  weapons,  political  and  psychological  considerations,  tentative 
courses  of  action  (CO  As),  and  a  proposed  planning  schedule. 

Air  Force  Doctrine  Document  2,  Organization  and  Employment  of  Aerospace 
Power,  provides  complementary  guidance  to  JPub  5-03.1  on  deliberate  and  crisis-action 
planning.  Campaign  plans  allow  theater  commanders  to  set  operational  tempo,  direct  the 
conduct  of  battles,  envision  objectives,  develop  concepts,  and  coordinate  logistics  to 
achieve  victory.  Planning  combat  operations  revolve  around  precise  communication  of 
commander’s  intent  and  a  shared,  clear  understanding  of  the  appropriate  operational 
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concepts  at  each  level  of  command.  The  Joint  Air  and  Space  Operations  Plan  (JASOP) 
and  supporting  plans,  developed  in  coordination  with  the  Joint  Air  Component 
Commander  and  Joint  Forces  Commander  (JFC),  state  how  the  air  commander  conducts 
theater  air  and  space  operations.  This  is  the  heart  of  what  is  colloquially  called  ‘the  air 
campaign.’  The  JASOP  is  developed  during  the  concept  development  and  plan 
development  phases  of  deliberate  planning,  as  the  foundation  of  an  OPLAN  or 
CONPLAN;  or  during  the  execution  planning  phase  of  crisis  action  planning,  in  concert 
with  overall  theater  campaign  planning.  Developing  the  JASOP  involves  a  five-stage 
planning  process,  with  each  stage  producing  a  product.  While  the  stages  are  presented  in 
sequential  order,  work  on  the  various  stages  can  be  either  concurrent  or  sequential.  The 
first  stage  of  JASOP  development  is  directly  relevant  to  this  research. 

Operational  environment  research  focuses  on  gaining  a  comprehensive 
understanding  of  the  entire  theater  of  operations,  the  adversary,  and  friendly  forces 
available  to  accomplish  the  JFC’s  objectives.  Issues  that  must  be  considered  during  this 
stage  include  available  forces,  command  relationships  (national  and  multinational),  threat 
and  force  protection  requirements,  rules  of  engagement,  applicable  treaties  and 
agreements,  base-use  rights,  and  overflight  rights.  Available  support  from  allies  and  the 
degree  of  political  and  social  stability  in  the  region  are  also  important  to  study. 
Additionally,  appropriate  logistics  information  is  acquired  concerning  what  is  available  in 
theater  and  what  is  provided  through  existing  ports,  depots,  war  reserve  materiel,  and 
host-nation  support.  Finally,  a  detailed  intelligence  picture  is  developed  that  includes 
indications  and  warning,  current  intelligence,  general  military  intelligence,  target 
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intelligence,  and  a  complete  analysis  of  enemy,  neutral,  and  friendly  forces  and  potential 
courses  of  action. 

Following  the  guidance  of  JPub  5-03.1  and  Air  Force  Doctrine  Document  2,  the 
specific  types  of  contextual  issues  examined  during  the  crisis  action  planning  process  are 
incorporated  into  an  initial  hierarchy  of  issues.  This  developed  hierarchy  of  issues, 
presented  as  Figure  1,  is  the  foundation  upon  which  this  research  is  conducted. 


FIGURE  1 :  Initial  value  hierarchy. 


Six  top-level  issues  are  identified,  each  of  which  is  discussed  in  both  JPub  5-03.1 
[JPub5-03.1,  1994;  pg  P-1  to  P-6-5]  and  AFDD  2  [AFDD  2,  1998;  pg  76],  The  US 
Forces  and  Host-Nation  issues  are  expanded  to  include  sub-issues  relevant  to  the 
capability  of  our  forces  to  conduct  combat  operations  in  the  theater. 
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SUMMARY 


This  chapter  reviews  current  Air  Force  and  Joint  documentation  on  planning 
factors  considered  when  selecting  combat  aircraft  to  deploy  in  response  to  a  crisis. 
Threats  can  vary  significantly;  therefore,  theater  commanders  must  be  able  to  tailor  their 
response  to  the  current  threat.  The  methodological  tools  and  techniques  appropriate  for 
this  research  are  also  reviewed.  Interviewing  techniques  based  upon  the  Cognitive  Task 
Analysis  theory,  a  tool  to  understand  the  cognitive  requirements  of  individuals  involved 
with  a  task  or  decision,  are  presented  and  explored.  Value  Focused  Thinking,  a 
methodology  to  assist  decision-makers  in  making  better  decisions,  are  also  reviewed. 
The  Delphi  Study  technique,  a  tool  for  eliciting  group  conformity  on  a  specific  topic  or 
subject,  is  presented  and  discussed.  The  next  chapter  describes  the  methodology  behind 
the  deployment  planning  factor  elicitation  interviews,  the  development  of  the  Value 
Focused  Thinking  decision  support  tool,  and  the  Delphi  study  used  to  confirm  the 
decision  support  tool  and  determine  weights  for  the  value  analysis. 
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III.  METHODOLOGY 


The  methodology  for  this  research  consists  of  three  phases,  as  shown  in  Table  2 


below. 


TABLE  2:  RESEARCH  METHODOLOGY  PHASES 

Phase  I 

Cognitive  Task  Analysis  Interviews 

Phase  II 

Value-Focused  Thinking  Decision  Support  Tool  Development 

Phase  HI 

Delphi  Study  Decision  Support  Tool  Confirmation 

Phase  I  of  this  research,  and  the  literature  review  on  campaign  planning,  are 
conducted  to  answer  research  question  1,  What  factors  other  than  absolute  aircraft 
capabilities  should  campaign  planners  consider  when  selecting  a  specific  set  of  combat 
aircraft  to  counter  a  theater  threat?  Research  questions  2  and  3,  How  are  these  factors 
qualified,  and  What  relationships  link  these  factors  to  each  other,  respectively,  are 
answered  by  Phase  II  of  this  research.  Phase  III  focuses  on  answering  research  question 
4,  Based  on  a  Southwest  Asia  scenario,  what  is  the  relative  importance  of  each  factor 
with  respect  to  the  others? 

Phase  I  applies  a  cognitive  task  analysis  (CTA)  to  experts  in  Air  Force  combat 
deployment  planning,  in  order  to  develop  a  comprehensive  list  of  planning  considerations 
or  decision  points  that  campaign  planners  must  consider  when  deploying  combat  aircraft. 
Phase  II  takes  the  information  gleaned  from  the  Phase  I  CTA  and  develops  the  decision 
points  into  a  quantitative,  value  focused  thinking  (VFT)  decision  support  tool.  This 
phase  is  conducted  by  interviewing  campaign  planners  and  staffers  at  Air  Force  Central 
Command.  Phase  III  uses  a  Delphi  study  to  obtain  agreement  from  the  individuals 
studied  on  the  factors  defined  and  develops  weights  of  importance,  relative  to  each  other. 
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within  the  VFT  decision  support  tool.  The  development  and  operation  of  these  three 
Phases  are  presented  within  this  chapter. 

PHASE  I  -  CTA  STUDY 

OBJECTIVES  OF  CTA 

The  objective  of  the  Cognitive  Task  Analysis  is  to  elicit  campaign-specific 
plaiming  considerations  or  factors,  not  tied  to  the  absolute  capabilities  of  individual 
USAF  combat  aircraft,  that  campaign  planners  and  theater  commanders  must  take  into 
account  when  groups  of  combat  aircraft  are  selected  to  deploy.  The  CTA  serves  to 
confirm  and  expand  the  Base  Value  Hierarchy  developed  through  the  literature  review  of 
Joint  and  Air  Force  campaign  planning  publications  and  doctrine.  This  expanded  value 
hierarchy  provides  the  basis  for  the  development  of  a  campaign-specific,  combat  aircraft 
selection,  decision  support  tool. 

OBJECTIVES  OF  COGNITIVE  DECISION  MAKING  USE 

The  Cognitive  Decision-Making  (CDM)  analysis,  discussed  in  Chapter  II,  is  used 
to  elicit  campaign-specific  issues  from  subject  matter  experts.  These  issues  can  be 
described  as  points  in  the  campaign  planning  process  where  critical  decisions,  decisions 
that  a  novice  would  have  sufficient  difficulty  overcoming,  may  be  required.  The 
presence  and  severity  of  these  issues  may  hamper  the  selection  of  a  specific  set  of  combat 
aircraft. 

JOB  DESCRIPTION,  PRIMARY  TASKS,  AND  TASKS  ANALYZED 

When  campaign  planners  perform  their  crisis  action  planning  function,  they  are 
often  constrained  by  time  and  operate  under  immense  pressure.  Crisis  action  planning 
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requires  a  significant  level  of  knowledge  and  understanding  of  the  planning  and 
deployment  process.  The  purpose  of  the  CTA  is  to  study  this  cognitive  portion  of  the 
planning  process  in  an  effort  to  provide  campaign  planners  with  a  tool  that  identifies 
possible  issues  and  constraints  that  may  plague  a  deployment  plan.  This  tool  can  help 
campaign  planners  to  maintain  awareness  of  these  issues  and  constraints,  whereas  they 
may  otherwise  go  unnoticed  during  the  development  of  the  plan  due  to  time  and 
operational  constraints. 

PARTICIPANT  SELECTION 

Twenty  Air  Force  officers  knowledgeable  in  deployment  operations,  through  both 
experience  and  training,  were  interviewed.  These  subject  matter  experts  (SMEs)  were 
selected  from  Aerospace  Operations  offices  of  Air  Combat  Command  and  Air  Staff, 
campaign  planning  instructors  from  Air  University,  and  campaign  planners  from  Air 
Force  Central  Command.  These  individuals  hold  the  rank  of  major  through  colonel;  all 
have  a  presumed  level  of  experience  in  deployments  and  planning. 

MATERIALS 

A  Bullet  Background  Paper  (BBP)  [Tongue  and  Quill,  1995;  pg  137]  is  developed 
as  a  pre-talker  for  the  interview.  This  paper  is  given  to  each  interviewee  at  the  start  of  the 
interview  and  is  briefly  discussed  to  acquaint  the  interviewee  to  the  subject.  The  BBP  is 
presented  as  Appendix  A. 

The  BBP  is  developed  following  the  CDM  interview  method  described  by  Klein 
et  al.  [Klein  et  al,  1997:  Pg  2].  Klein  discusses  how  CDM  interviews  need  to  focus  in  on 
a  specific  task  or  set  of  tasks.  The  course  of  the  interview  should  move  from  a  general 
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overview  of  the  task  or  set  of  tasks  to  specific  details  on  the  cognitive  requirements  of  the 
individual  task  or  tasks.  The  BBP  is  developed  in  order  to  provide  the  interviewee  with  a 
brief  description  of  the  basis  for  this  research,  an  overview  of  how  the  interview  is 
conducted,  the  objectives  of  the  interview,  based  upon  Klein  et  al.  methodology,  and  to 
provide  eontact  information  for  the  research.  The  description  section  discusses  the 
evolution  of  the  Expeditionary  Air  Foree  concept  and  how  this  research  came  to  light. 

The  interview  overview  section  provides  the  interviewee  with  a  focus  for  the  questions 
that  are  asked  during  the  interview.  A  review  of  this  section  assists  the  interview  process 
by  focusing  the  interviewee  on  the  areas  of  knowledge  and  expertise  that  are  relevant  to 
the  interview.  The  final  contact  section  of  the  BBP  provides  the  interviewee  with 
information  on  how  to  contact  the  researcher  if  the  interviewee  has  any  additional 
information  or  questions  regarding  the  interview  or  the  research  in  general. 

A  narrative  for  the  interview  is  developed  to  assist  the  interviewer  in  conducting 
the  interview.  The  narrative  is  developed  in  the  image  of  Klein  et  al.  Methodology,  and 
provides  the  interviewer  with  a  summary  of  the  important  points  to  hit  during  the 
interview  as  well  as  a  short  list  of  probes  to  use  to  elicit  knowledge  and  understanding  of 
the  decision  points.  The  interview  narrative  is  presented  as  Appendix  B.  The  narrative  is 
developed  to  assist  the  interviewer  in  maintaining  the  specific  focus  of  the  interview 
goals,  which  is  to  identify  the  non-technically  driven  issues  that  campaign  planners  have 
historically  considered  when  selecting  combat  aircraft  to  respond  to  a  theater  crisis. 

A  base  value  hierarchy,  used  during  the  interview,  is  developed  using  the  initial 
value  hierarchy  presented  in  Figure  1  of  Chapter  1  of  this  document  and  through 
discussions  during  OPER  649,  Advanced  Decision  Analysis,  at  the  Air  Force  Institute  of 
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Technology  (AFIT).  This  base  value  hierarchy  is  presented  as  Figure  2.  The  objective  of 
the  OPER  649  class  is  to  develop  the  Value  Focused  Thinking  Decision  Support  tool, 
which  is  discussed  in  Phase  II,  Value  Focused  Thinking  Decision  Support  Tool.  LTC 
Jack  Kloeber,  USA,  conducts  the  course.  During  this  course,  several  evolutions  of  the 
initial  Campaign-specific  Issue  value  hierarchy  led  to  the  development  of  the  CTA  study 
value  hierarchy  shown  in  Figure  2. 

Comparing  the  initial  value  hierarchy  and  the  CTA  study  value  hierarchy,  many 
obvious  differences  are  noted.  First,  the  CTA  study  value  hierarchy  is  developed  to 
better  represent  the  motivation  behind  the  value  hierarchy;  many  separate  issues  must  be 
considered  in  order  to  determine  the  aircraft  asset  set  that  provides  the  best  value,  or 
worth,  to  the  theater  commander.  Host-Nation  and  Multi-National  Politics,  given  top- 
level  values  in  Figure  1,  have  been  incorporated  together  in  Figure  2. 


Campaign  Specific  Issue  Hierarchy 


FIGURE  2;  CTA  study  base  value  hierarchy. 
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The  Host-Nation  and  Multi-National  Politics  issues  are  combined  because  it  is  believed 


that  they  are  similar  in  their  focus  on  the  location  of  the  theater.  The  Staging  Base  value 
is  expanded  based  upon  the  civil  engineering  experience  of  the  researcher,  and  through 
discussions  in  the  OPER  649  course.  At  this  point,  the  Airfield  issues:  Runway  Length 
and  Weight  capacity,  as  well  as  Ramp  Space,  are  identified  as  constraints.  These  issues 
may  prohibit  a  particular  set  of  assets  from  bedding  down  at  a  pre-determined  staging 
base  if  the  staging  base  cannot  support  the  physical  characteristics  of  the  aircraft  in  the 
asset  set.  Munitions  Storage  Area  (MSA)  and  Petroleum,  Oils,  and  Lubricants  (POL) 
issues  are  added  based  upon  their  importance  to  operational  requirements  at  the  pre¬ 
determined  staging  base.  Depending  upon  the  capabilities  of  the  staging  base  with 
respect  to  MSA  and  POL  issues,  the  operational  capability  of  the  combat  aircraft  within  a 
unique  asset  set  may  be  hampered.  Location  characteristics  are  also  expanded  to  provide 
a  better  description  of  the  types  of  issues  studied.  Through  OPER  649  discussions,  as 
well  as  discussions  with  Major  Alan  Johnson,  USAF,  and  Major  Steve  Swartz,  USAF, 
both  assistant  professors  at  AFIT,  it  is  determined  that  both  topography  and  weather  or 
climate  characteristics  at  the  location  of  the  pre-determined  staging  base  may  be 
important  issues.  Based  upon  the  analysis  of  the  initial  value  hierarchy,  the  US  Forces 
factor  is  removed  from  the  CTA  study  hierarchy.  The  Operational  Capabilities  factor  is 
a  direct  analysis  of  the  absolute  capabilities  of  the  combat  aircraft,  which  is  outside  the 
scope  of  this  research.  Therefore,  this  factor  is  re-located  to  the  top  tier  values  on  the 
CTA  study  hierarchy,  adjacent  to  the  Mitigating  Factors.  Mission  Goodness  is  defined 
under  this  study  as  the  absolute  capabilities  of  each  individual  type  of  fighter  and  bomber 
aircraft  to  complete  specific  aerospace  missions  defined  by  both  Air  Force  Manual  1-1 
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and  Air  Force  Doctrine  Document  2-1.  Mobility  resources,  important  to  the  capability  of 
USAF  forces  to  complete  specific  aerospace  missions,  are  also  outside  the  scope  of  this 
research.  Therefore,  this  issue  is  removed  from  the  CTA  study  hierarchy.  Mobility 
resources  will  be  specifically  studied  in  the  larger  scope  of  the  AFIT  and  Defense 
Advanced  Research  Projects  Agency  research  with  the  Advanced  Logistics  Project. 

During  the  CTA  interviews,  the  value  hierarchy  is  used  only  to  motivate  the 
interview  if  the  interviewer  feels  that  the  interview  is  not  progressing  satisfactorily.  Poor 
interview  progress  is  of  primary  concern  if  the  interviewee  either  does  not  understand  the 
issues  important  to  the  research,  the  interviewee  can  not  think  of  any  issues  early  in  the 
interview,  or  the  interviewee  is  not  responsive  to  the  questions  as  they  are  presented.  The 
value  hierarchy  that  is  used  for  these  interviews  is  presented  as  Figure  2. 

CTA  PROCEDURES 

The  interview  is  developed  using  the  “Interviewer’s  Manual”  [Interviewer’s 
Manual:  pg  15]  and  CDM  interview  technique  [Klein  et  al.,  1997:  pg  2].  Probes  are  used 
to  elicit  insightful  information  from  the  subject  matter  experts  (SMEs).  The  use  of 
probes  in  unstructured  interviews  serves  two  major  functions.  First,  probes  help  to 
motivate  the  interviewee  to  communicate  more  fully,  enabling  the  interviewee  to  enlarge 
upon,  clariiy,  or  explain  the  reasons  behind  what  is  said.  Secondly,  probes  help  the 
interviewee  to  focus  on  the  specific  content  of  the  interview  so  that  irrelevant  and 
uimecessary  information  can  be  avoided.  The  interviewer’s  most  important  concern  is  to 
ensure  that  probes  are  used  without  introducing  bias  into  the  interviewee’s  discussion  of 
the  issues.  The  Interviewer’s  Manual  [Interviewer’s  Manual:  pg  15-18]  and  numerous 
articles  on  CTA  and  CDM  discussed  in  this  paper  provide  possible  probes  useful  in  CDM 
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analyses.  Probes,  as  detailed  and  presented  by  Klein  et  al.,  [Militello,  1998:  pg  1622]  are 
used  during  the  CDM  interviews.  An  interview  script,  discussed  earlier,  is  used  to  assist 
the  interviewer  during  the  interview  and  to  ensure  the  interview  itself  was  kept  on  track 
with  the  research  goals. 

The  interviews  follow  a  modified  CDM  methodology.  The  interviewees  are  first 
presented  with  a  Southwest  Asia  Theater,  Air  Expeditionary  Force  deployment  scenario. 
The  interviewees  are  then  asked  to  speak  to  their  knowledge  and  understanding  about 
deploying  combat  aircraft  under  this  scenario.  This  allows  the  interviewee  to  become 
acclimated  to  the  topic  of  discussion  and  to  the  level  of  operational  experience  and 
knowledge  polled.  During  this  portion  of  the  interview,  the  interviewee  is  allowed  to 
relate  the  problem  to  situations  that  he  or  she  has  experienced  first-hand.  Comments 
based  upon  hearsay  are  ignored.  Using  the  modified  CDM  methodology,  the  interviewer 
should  try  to  keep  the  interviewee  motivated  to  continue  his  or  her  discourse  throughout 
the  interview.  Before  individual  interviews  are  concluded,  the  interviewer  should  attempt 
to  obtain  clarification  on  points  that  may  have  only  been  previously  touched  on,  confirm 
or  deny  unclear  comments,  and  elicit  any  further  comments. 

PHASE  II  -  VALUE  FOCUSED  THINKING  STUDY 

OBJECTIVES 

The  Value  Focused  Thinking  (VFT)  study  holds  two  purposes.  The  first  is  to 
validate  and  verify  the  information,  obtained  through  the  interviews  held  as  a  part  of  the 
Cognitive  Task  Analysis  (CTA)  study,  with  current  Southwest  Asia  (SWA)  campaign 
planners.  The  second  purpose  is  to  define  a  quantifiable  range  of  values  for,  or  if  not 
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quantifiable  identify  as  a  constraint,  each  of  the  campaign-specific  issues.  Through  the 
two  components  of  this  phase,  a  more  complete  hierarchy  of  values  can  be  developed. 
This  value  hierarchy  is  incorporated  into  a  spreadsheet-based  VFT  decision  support  tool. 

PARTICIPANT  SELECTION 

Current  campaign  plarmers  from  Headquarters,  USAF  Central  Command 
(CENTAF)  are  participating  in  the  construction  of  the  decision  support  tool.  As  the 
research  is  focusing  on  a  SWA  contingency  deployment,  it  is  believed  that  CENTAF 
planners  have  the  required  knowledge  and  experience  to  act  as  the  decision-maker 
representatives. 

MATERIALS  USED 

A  dictionary  of  campaign-specific  issues  assists  the  VFT  study.  This  dictionary 
presented  as  Appendix  C  is  important  to  maintaining  consistency  in  the  description  of  the 
campaign-specific  issues  and  as  to  how  these  issues  are  measured.  These  definitions  are 
adjusted  if,  during  the  VFT  study,  the  interviewees  feel  that  different  definitions  better 
represent  the  views  and  requirements  of  the  theater  commander. 

An  example  graphical  representation  of  the  measurement  of  the  campaign- 
specific  issues  is  presented  as  Figure  3.  A  single,  blank  graph  is  used  for  each  campaign- 
specific  issue  prior  to  the  interview.  For  each  issue,  X  and  Y-axes  are  presented.  The  X- 
axis  represents  the  measurement  upon  which  a  specific  aircraft  asset  or  asset  set  is 
measured.  The  Y-axis  represents  the  transformed  value  or  worth  to  the  commander.  For 
each  issue,  regardless  of  the  type  and  range  of  measurement,  the  Y-axis  represents  value, 
ranging  from  0  to  1 .  The  individual  issue  X-axes  are  quantified  on  how  the  individual 
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issues  can  be  measured.  The  graphical  representation  shown  in  Figure  3  presents  how  the 
decision-maker  views  the  transformation  of  measurement  to  value.  Note  that  the 
presented  linear  graph  is  just  an  example. 


Discussions  during  OPER  649,  and  with  ETC  Kloeber,  Major  Johnson,  Major  Swartz  and 
current  AFIT  graduate  students  aided  in  developing  these  measurements.  The  individual 
campaign-specific  issue  graphs  are  presented  as  Appendix  D.  Since  operational 
capability  is  the  main  focus  behind  these  issues,  degradation  to  mission  capability  is  used 
as  the  measurement  for  many  of  the  issues. 

These  blank  graphs,  presented  in  Appendix  D,  are  individually  printed  onto  paper 
and  placed  in  a  clear  plastic  cover.  The  plastic-protected  graphs  are  placed  into  a  three- 
ring  binder.  This  medium  facilitates  the  smooth  transition  between  graph  presentations, 
and  allows  the  decision-maker  representative  to  re-think  functions  previously  developed 
without  wasting  time. 

To  quickly  familiarize  the  study  participants  to  VFT  methodology,  a  short 
example  of  buying  a  car  was  developed  and  described.  This  example  is  presented  as 
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Appendix  E.  This  example  is  used  to  provide  a  basic  level  of  understanding  to  the 
CENTAF  decision-makers  on  VFT  methods. 

PROCEDURES 

A  group  meeting  is  arranged  with  campaign  planners  from  CENTAF.  At  this 
meeting,  the  campaign  planners  are  the  decision-makers  in  the  development  of  the 
campaign-specific  issue  measurements.  A  brief  description  of  the  purpose  of  this 
research  and  a  background  of  VFT  is  presented.  The  updated  value  hierarchy  is  then 
presented  to  facilitate  feedback  and  agreement  upon  the  value  hierarchy  as  it  has  been 
developed  to  this  point.  Once  acceptance  is  achieved  or  the  correct  changes  are  made, 
the  VFT  interview  can  begin. 

The  purpose  of  the  VFT  interview  is  to  elicit  a  range  of  measurement  for  each  of 
the  campaign-specific  issue  measures  in  the  value  hierarchy.  Campaign-specific  issue 
constraints  are  not  be  discussed  at  this  interview,  due  to  the  fact  that  these  issues  have 
been  developed  as  either  “yes”  or  “no”  values.  Each  campaign-specific  issue  measure  is 
presented  individually,  allowing  the  decision-makers  to  focus  on  how  each  issue  is 
measured  specifically.  A  pre-determined  range  of  measurement  for  each  issue  is  first 
presented,  as  shown  as  the  X-axes  in  the  graphs  shown  in  Appendix  D.  This  provides  a 
starting  point  for  discussion  with  the  decision-makers,  allowing  them  to  change  or  alter 
the  measurement  range  as  needed.  Once  the  X-axis  measurement  was  confirmed,  the 
measurement  scores  that  would  result  in  the  lowest  and  highest  values  (Y-axis) 
respectively  were  presented  to  the  interviewee.  These  points  were  marked  on  the  plastic 
protective  sheet  using  a  Vis-a-vis  marker.  The  interviewee  was  then  asked  at  what  score 
of  measurement  would  the  campaign  specific  issue  begin  to  substantially  lose  value  (i.e. 
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at  what  point  along  the  X-axis  would  the  value  of  the  campaign  specific  issue  drop  from 
the  high  value  of  1 .0).  After  marking  this  point  on  the  plastic-protected  graph,  the 
interviewee  was  then  asked  to  identify  the  point  at  which  the  campaign  specific  issue 
begins  to  substantially  increase  in  value,  from  the  low  value  of  0.0.  This  point  was  also 
identified  on  the  graph  using  the  Vis-a-vis  marker.  The  interviewee  was  then  asked  to 
“connect  the  dots”,  visually  describing  how  the  scoring  of  the  individual  campaign 
specific  issue  would  translate  to  value  to  the  commander.  After  a  monotonically 
increasing  or  decreasing  function  was  satisfactorily  drawn  on  the  graph,  the  interviewer 
was  asked  if  he  was  satisfied  with  the  function.  The  graphical  response  shown  in  figure  3 
is  an  example  of  what  the  interviewee  may  identify  as  the  function,  but  exponential  or 
step  functions  may  also  occur.  This  process  was  continued  until  a  function  was  identified 
for  eaeh  of  the  campaign  specific  issues. 

Once  ranges  of  measurement  have  been  obtained,  value  functions  are  elicited  via 
classical  VFT  interview  techniques,  techniques  of  whieh  have  been  developed  by  Keeney 
and  modified  by  Kirkwood  [Kirkwood,  1997;  pg  300-320].  The  final  product  of  this 
VFT  interview  is  a  range  of  measure  and  single  dimension  value  function  for  each 
measurable  campaign-specific  issue  presented  in  the  value  hierarchy. 

PHASE  III  -  DELPHI  STUDY 

OBJECTIVES  OF  DELPHI  STUDY 

The  purpose  of  this  Delphi  study  is  to  elicit  relative  weights  of  the  individual 
campaign  specific  issues  identified.  Campaign  planning  SMEs  contacted  during  the  CTA 
and  VFT  interviews  are  asked  to  partieipate  in  the  Delphi  study.  The  25  USAF  officers 
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that  comprise  the  DELPHI  study  pool  are  listed  as  Appendix  J.  Two  rounds  of 
questiormaires  are  conducted,  with  two  questionnaires  per  round. 

Delphi  Study  Round  One  (questionnaires  #1  and  #2): 

There  are  three  purposes  to  the  first  round  of  questionnaires: 

1)  Reacquaint  the  SMEs  to  the  purpose  and  aim  of  the  Delphi  study  and  the 
overall  research  focus 

2)  Present  and  confirm  the  updated  value  hierarchy,  shown  as  Figure  4 

3)  Provide  a  first  attempt  at  converging  to  a  consensus  the  relative  ranking  of 
individual  campaign  specific  issues 

Delphi  Study  Round  Two  (questionnaires  #3  and  #4): 

There  are  two  purposes  to  the  second  round  of  questionnaires: 

1)  Continue  diseussion  on  the  validity,  content,  and  organization  of  the  value 
hierarehy,  making  changes  warranted  from  first  round  of  Delphi  study 

2)  Determine  the  individual  weighting  of  each  Campaign  Specific  Issue  as  they  fit 
into  the  value  hierarchy 

PARTICIPANT  SELECTION 

Participants  from  the  CTA  interviews  and  VFT  decision  support  tool  construction 
are  invited  to  partieipate  in  the  study.  Each  of  these  individuals  has  a  general 
understanding  of  the  goal  of  the  research  and  has  expertise  in  campaign  planning  and 
deployment.  It  is  therefore  believed  that  the  SMEs  can  provide  useful  feedback  and 
continued  guidanee  on  the  value  hierarchy,  decision  support  tool,  and  decision  point 
weightings. 
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DELPHI  STUDY  DESIGN 

The  purpose  of  the  Delphi  study  is  to  elicit  an  agreement,  between  the  SMEs  used 
in  the  CTA  analysis,  on  the  VFT  decision  support  tool.  The  Delphi  study  is  also  used  to 
elicit  weights  for  each  of  the  campaign-specific  issues  within  the  value  hierarchy.  This 
study  is  being  conducted  via  electronic  mail.  Phone  interviews  are  conducted  only  if 
specifically  requested  by  the  individual  SME. 

Questionnaire  #1  in  the  Delphi  study  provides  the  SMEs  an  introduction  to  the 
Delphi  technique,  as  well  as  a  review  of  the  research  focus.  The  questions  asked  of  the 
experts  within  questionnaire  #1  are  dedicated  to  the  confirmation  of  the  value  hierarchy 
produced  through  interviews  with  the  SMEs  themselves,  as  well  as  a  confirmation  of  the 
decision  support  tool  as  it  was  developed  through  the  aid  of  CENTAF  campaign  planners. 
Questionnaire  #1  also  asks  the  SMEs  to  rank  order  the  campaign-specific  issues  based 
upon  their  ranges  of  measurement.  This  questionnaire  is  presented  as  Appendix  F.  Both 
questionnaire  #1  and  #2  are  electronically  mailed  (e-mailed)  to  all  SME  participants  in  a 
group  mailing.  The  answers  provided  from  questionnaire  #1  are  compiled  and  presented 
in  questionnaire  #2.  The  Delphi  study  takes  several  series  of  questionnaires  to  come  to 
an  actual  agreement  on  the  questions  asked.  Each  successive  questionnaire  also  provides 
a  presentation  of  the  comments  made  on  the  previous  questionnaire. 

Questionnaires  #3  and  #4,  comprising  the  second  round  of  the  Delphi  study,  ask 
the  participants  to  identify  a  numerical  weight  for  each  of  the  campaign-specific  issues, 
relative  to  and  consistent  with  the  rank  ordering  they  determined  in  the  first  round  of 
questionnaires.  The  second  round  of  the  Delphi  Study  seeks  to  determine  the  relative 
weightings  of  the  individual  campaign  specific  issues  as  they  related  to  each  other  in  the 
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value  hierarchy.  It  should  be  carefully  noted  that  these  weightings  have  been  focused  on 
an  Air  Expeditionary  Force  (AEF)  deployment  to  the  Southwest  Asian  (SWA)  Theater, 
in  which  the  primary  purpose  of  the  deployment  is  to  conduct  crisis-action  response 
combat  actions.  It  is  hypothesized  that  a  deployment  to  a  different  theater  for  a  different 
purpose  would  lead  to  a  different  set  of  weightings  on  the  campaign  specific  issues.  It 
should  also  be  noted  that  Campaign  Specific  Constraints  are  not  discussed  in  this 
questionnaire,  although  they  were  ranked  for  level  of  importance  in  the  previous  round  of 
the  Delphi  study.  In  determining  weights,  constraints  do  not  apply  because  they  either 
completely  impact  the  alternative  asset  set,  or  they  do  not  at  all  impact  the  alternative 
asset  set.  As  constraints  are  either  go  or  no-go,  weightings  do  not  apply. 

The  process  of  eliciting  the  relative  weights  is  made  easier  through  the  ranking  of 
the  campaign  specific  issues  during  the  first  round  of  the  Delphi  study.  This  phase  of  the 
study  is  conducted  in  two  separate  but  equivalent  methods.  The  first  method  is  to 
develop  questionnaire  #3  for  e-mail  delivery  and  response.  The  second  method  of  this 
phase  of  the  study  is  to  interview  study  group  members  in  person.  Questionnaire  #4  in 
the  Delphi  study  confirms  the  individual  campaign  specific  issue  weights  and  concludes 
the  study. 

Questionnaire  #3  has  one  question.  This  question  focuses  on  the  relative 
weighting  of  the  campaign  specific  issues  as  they  relate  to  each  other.  Based  upon  the 
rankings  provided  from  the  previous  two  questionnaires,  the  highest-ranking  sub-issue 
within  each  parent  issue  is  compared  to  the  other  sub-issues  within  that  parent  issue.  By 
comparing  these  highest-ranking  sub-issues,  the  relative  weightings  of  the  parent  issues 
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can  be  obtained  with  respect  to  each  other.  To  compare  the  sub-issues,  the  following 
question  is  asked  for  each  pair  of  sub-issues. 

Please  state  how  many  times  more  important  the  swing  from  best  score  to  worst 
score  for  the  issue  in  Column  ’A  ’  is  than  the  swing  from  best  score  to  worst  score  for  the 
issue  in  Column  ‘B 

The  question  above  compares  the  entire  range  of  Issue  Y  measurement  to  the 
entire  range  of  Issue  X  measurement.  This  comparison  of  range  can  be  easily  transferred 
into  a  loss  of  value,  as  the  range  of  measurement  for  each  issue  translates  to  a  total  value 
of  1 .  This  question  therefore  asks  the  survey  respondent  to  quantify  the  difference  in 
value  for  each  two  compared  issues.  For  example,  if  the  issue  in  Column  ‘A’  were 
described  to  be  twice  (2x)  the  importance  than  the  issue  in  Column  ‘B,’  then  the  relative 
weight  of  the  issue  in  Column  ‘A’  is  twice  (2x)  the  relative  weight  of  the  issue  in  Column 
‘B’.  The  same  analysis  holds  true  for  the  relative  weighting  of  parent-issues  with  respect 
to  each  other.  Questionnaire  #3  is  presented  as  Appendix  G. 

Delphi  study  pool  members  from  Headquarters  USAF  Air  Staff  (HQ/US  AF),  Air 
Combat  Command,  and  USAF  Headquarters  Central  Command  (CENTAF)  are 
interviewed  personally  to  elicit  their  knowledge  and  opinions  on  the  relative  weightings 
between  the  campaign  specific  issues.  Interview  participants  are  e-mailed  a  brief 
description  of  the  purpose  of  the  interview  and  a  simple  example  of  what  relative  weights 
are  and  how  they  are  going  to  be  elicited  from  the  participant  during  the  interview.  This 
e-mail  is  presented  as  Appendix  L.  At  the  beginning  of  the  interview,  the  interviewee  is 
asked  if  they  have  any  questions  about  the  purpose  of  the  interview,  what  the  weighting 
of  the  issues  mean,  and  the  process  in  which  the  interview  is  conducted.  The  interviewee 
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is  also  instructed  to  focus  answers  based  upon  a  crisis-action,  AEF  deployment  to  the 
SWA  theater  in  which  the  primary  purpose  of  the  deployment  was  to  conduct  combat 
operations. 

As  with  the  e-mail  questionnaire,  the  interviewees  are  asked  to  individually 
compare  two  separate  campaign  specific  issues  to  determine  the  relative  weighting  of 
each.  Unlike  the  e-mail  questioimaire,  the  interviewees  are  given  a  visual  diagram  of  the 
range  of  measurement  of  each  issue  compared,  and  are  given  a  range  of  measurement  in 
which  each  issue  is  compared.  A  typical  setup  of  the  weight  elicitation  process  is 
presented  as  Figure  4. 

Issue  ‘A’  is  compared  to  Issue  ‘B’  along  their  ranges  of  measurement.  The 
interviewee  is  first  presented  with  the  ranges  of  measurement,  which  are  typically  only  a 
portion  of  the  full  range  of  measurement  obtained  by  CENTAF  during  the  Value  Focused 
Thinking  interviews.  The  interviewee  is  then  presented  with  two  alternatives,  an  example 
of  which  is  shown  in  Figure  4,  where  campaign  specific  issues  ‘A’  and  ‘B’  is  compared. 


Issue  A 


CAMPAIGN  VALUE  1  0 

SPECIFIC  ◄ - ► 

ISSUE  'A'  RANGE  Best  Worst 

Measurement  of  Issue  A 

VS. 

Issue  B 

CAMPAIGN  VALUE  1  0 

SPECIFIC  ^ ► 

ISSUE  'B*  RANGE  Best  Worst 


Measurement  of  Issue  B 

FIGURE  4:  Weight  comparison  of  Issue  ‘A’  and  Issue  ‘B’. 

The  interviewee  is  instructed  that  for  both  alternatives,  all  other  campaign  specific 
issues  and  aspects  of  the  deployment  should  be  considered  equivalent  to  both 
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alternatives.  This  means  that  looking  at  both  alternatives,  the  only  two  things  that  are 
different  about  the  alternatives  are  with  respect  to  issues  ‘A’  and  ‘B’.  Based  upon  the 
previous  round  of  the  Delphi  study,  issue  ‘A’  was  ranked  higher  in  importance  than  issue 
‘B’,  and  are  therefore  discussed  to  determine  the  portion  of  the  range  of  measurement  of 
issue  ‘A’  that  equaled  the  entire  range  of  measurement  of  issue  ‘B’.  The  alternatives  are 
then  discussed.  For  the  first  alternative,  issue  ‘A’  has  a  very  high  measurable  score 
(highest-value  based  upon  the  identified  range  of  measurement).  Issue  ‘B’,  for  this  first 
alternative,  has  a  very  low  measurable  score  (lowest-value  based  upon  the  identified 
range  of  measurement).  For  the  second  alternative.  Issue  ‘B’  has  a  very  high  measurable 
score  (highest-value  based  upon  the  identified  range  of  measurement).  The  following 
question  and  statement  are  then  proposed: 

If,  for  this  second  alternative,  Issue  ‘B  ’  could  command  a  very  high  value,  what 
loss  of  Issue  ‘A  ’  would  you  be  willing  to  accept  to  be  indifferent  to  selecting  either 
alternative?  At  this  score  of  Issue  A  ’,  if  Issue  A  ’  had  a  higher  score  (higher  value),  you 
would  always  select  the  second  alternative;  if  Issue  A  ’  had  a  lower  score  (lower  value), 
you  would  always  select  the  first  alternative.  For  these  two  alternatives,  all  other  issues 
are  held  constant. 

Sixteen  comparison  questions  are  asked  to  determine  the  relative  weighting 
between  parent  issues  and  between  sub-issues  within  each  parent  issue.  This  number  of 
questions  is  needed  to  properly  compare  sub-issues  within  a  particular  parent  issue,  and 
to  compare  parent  issues  to  each  other.  As  with  the  e-mail  questionnaire,  the  parent 
issues  are  determined  by  comparing  the  two  highest-ranking  issues,  as  determined  by  the 
first  round  of  the  Delphi  study. 
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Once  the  relative  weights,  based  upon  ranges  of  measurement,  of  the  campaign 
specific  issues  are  obtained  from  the  interviewees,  the  weights  are  transformed  to  reflect 
the  value  of  each  campaign  specific  issue.  Using  the  model,  the  value  of  the  high  and 
low  range  of  measurement  for  each  issue  is  ealculated.  The  analysis  of  the  interview  data 
is  then  based  upon  comparing  values,  and  the  information  obtained  from  the  e-mail 
questionnaires  was  added  to  the  data.  To  determine  the  relative  weights  for  eaeh  issue 
within  a  particular  parent  issue,  the  relative  weighting  of  each  sub-issue  must  sum  to  1 ,  as 
shown  in  Equation  2.  The  comparisons  of  individual  sub-issues  within  a  unique  parent 
issue  allow  an  equation  to  be  developed  that  contains  only  1  sub-issue.  An  example  of 
this  is  the  Enemy  parent  issue.  Two  questions  in  both  the  interview  elicitation  and  the  e- 
mail  questionnaire  are  dedicated  to  the  three  sub-issues  of  the  Enemy  parent  issue.  The 
first  question  compares  USAF  v^.  Enemy  Capability  to  Enemy  ISR,  and  the  second 
question  compares  USAF  vi'.  Enemy  Capability  to  Enemy  Interference.  Looking  at 
Equation  2,  it  is  held  that  the  relative  weights  of  USAF  vs.  Enemy  Capability,  Enemy  ISR, 
and  Enemy  Interference  must  sum  to  1 .  Beeause  Enemy  Capability  and  Enemy  ISR  are 
both  compared  to  USAF  vs.  Enemy  Capability,  the  variables  in  Equation  2  are  reduced  to 
only  the  USAF  vs.  Enemy  Capability  issue.  Onee  the  relative  weight  of  the  USAF  vs. 
Enemy  Capability  issue  is  determined,  the  comparisons  are  used  to  determine  the  relative 
weights  of  both  Enemy  ISR  and  Enemy  Capability.  This  process  is  repeated  to  determine 
the  relative  weights  for  each  of  the  sub-issues  within  a  particular  parent  issue. 

Parent  issue  weights  are  determined  using  the  same  method.  Using  Equation  2, 
the  parent  issues  must  sum  to  1 .  Based  upon  the  Campaign  Specific  Issue  value 
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hierarchy  updated  through  the  first  round  of  the  Delphi  study,  only  three  comparisons  are 
needed. 

To  complete  the  Value  Focused  Thinking  (VFT)  Campaign  Issue  analysis  model, 
the  relative  weights  of  the  sub-issues  must  be  transformed  into  global  weights.  Global 
weights  are  the  true  level  of  importance  that  a  single  issue  has  on  the  overall  analysis  of  a 
particular  asset  set.  Relative  weights  of  sub-issues  provide  the  portion  of  the  parent 
issue’s  weight  that  affects  the  calculation.  To  transform  relative  weights  to  global 
weights,  the  relative  weight  of  a  particular  sub-issue  is  multiplied  by  the  relative  weight 
of  the  parent  issue  that  the  particular  sub-issue  falls  into.  A  set  of  global  weights  is 
calculated  based  upon  each  individual  interview  and  e-mail  questionnaire  response  given. 

The  second  round  of  questionnaires  is  complete  when  a  consensus  of  the  SMEs  is 
obtained  on  the  numerical  values  of  relative  weights  for  each  individual  campaign- 
specific  issue.  With  the  development  of  relative  weights,  the  campaign-specific  decision 
support  tool  is  complete. 

SENSITIVITY  ANALYSIS  OF  ISSUE  WEIGHTS 

Once  the  VFT  model  is  developed,  it  can  be  used  to  conduct  a  sensitivity  analysis 
to  determine  the  impact  of  the  ranking  of  alternatives  based  upon  changes  in  the  relative 
weighting  of  the  parent  issues.  These  parent-issue  weights  represent  the  relative 
importance  that  is  attached  to  the  individual  issues  within  the  parent  issue,  and  may  be  a 
topic  of  disagreement  between  the  subject  matter  experts.  To  conduct  the  sensitivity 
analysis  the  weight  of  a  single  parent  issue  will  be  evaluated,  while  holding  the  other 
weights  to  the  same  ratio  as  obtained  from  the  Delphi  study.  All  of  the  weights  must  sum 
to  1 .  In  his  textbook,  Kirkwood  describes  the  method  of  varying  all  other  weights  in  the 
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hierarchy  [Kirkwood,  1997:  pg  84].  By  varying  the  weight  studied  over  a  range  of 
possible  values  for  the  weights,  the  sensitivity  of  the  ranking  of  assets  sets  to  the  specific 
weight  can  be  determined.  While  it  is  important  to  determine  the  sensitivity  across  the 
range  of  weights  obtained  from  the  SMEs,  it  is  also  important  to  understand  how  the 
rankings  may  change  based  upon  weights  outside  of  this  identified  range.  If  the  ranking 
of  the  asset  sets  do  not  change  within  the  range  of  weights  provided,  confidence  in 
defined  value  for  the  studied  weight  is  achieved.  Multiple  changes  in  rank  within  the 
range  of  weights  provided  suggest  that  there  is  disagreement  between  the  SMEs  as  to  the 
actual  importance  of  this  issue. 

SUMMARY 

This  chapter  details  the  development  of  the  three  stages  of  research  on  campaign- 
specific  issues.  First,  the  development  of  a  CTA  to  elicit  campaign-specific  issues  from 
deliberate  and  crisis  action  planning  subject  matter  experts  was  discussed.  The 
development  and  setup  of  a  Value-Focused  Thinking  interview  with  crisis  action 
planning  staff  members  at  Headquarters,  United  States  Air  Force  Central  Command  is 
diseussed.  The  Value-Focused  Thinking  interview  obtains  quantitative  ranges  of 
measurement  for  the  measurable  campaign-specific  issues,  within  the  context  of  a  crisis 
action  plan  in  the  Southwest  Asian  Theater  of  operations.  Finally,  the  development  and 
setup  of  a  Delphi  study  to  obtain  confirmation  of  the  developed  eampaign-speeific  issues 
hierarchy  and  to  determine  relative  weights  of  those  campaign-specific  issues  within  the 
Southwest  Asia  Theater  is  discussed. 

The  next  ehapter  discusses  the  results  obtained  from  the  CTA,  VFT  interview,  and 
Delphi  study.  The  integration  of  these  three  studies  lead  to  the  development  of  a 
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campaign-specific  decision  support  tool  to  aid  crisis  action  planners  in  assessing  the 
potential  impact  of  non-technical  issues  on  the  selection  and  deployment  of  a  specifie  set 
of  combat  aircraft. 
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IV.  RESULTS 


PHASE  I - CTA  STUDY 

During  the  CDM  interview  process,  individual  interviews  lasted  between  one  and 
two-and-one-half  hours,  depending  upon  the  level  of  interest  and  quantity  of  discussion 
provided  by  the  interviewee.  As  the  interviews  were  largely  unstructured,  it  was  felt  that 
setting  a  limit  of  the  interview  time  could  lead  to  missing  out  on  potentially  valuable 
information. 

All  of  the  subject  matter  experts  (SMEs)  contacted  were  very  supportive  of  the 
research  and  took  the  time  not  only  to  undergo  the  interview  but  to  understand  the  overall 
objectives  of  the  research.  At  the  conclusion  of  each  group  (Air  Combat  Command,  Air 
Staff,  and  Air  University)  of  interviews,  hand- written  dictation  of  the  interviews  were 
transcribed  into  electronic  media.  Interview  notes  from  these  interviews,  with  names 
omitted,  are  presented  as  Appendix  H. 

Campaign  Specific  Issues: 

At  the  completion  of  all  interviews,  the  interview  notes  were  reviewed  to 
determine  relevant  campaign  specific  issues.  The  issues  repeatedly  identified  through  the 
interviews  with  the  SMEs  were  incorporated  into  the  value  hierarchy.  The  updated  value 
hierarchy  is  presented  as  Figure  5. 

From  the  CTA  interviews,  descriptions  of  the  campaign  specific  issues  were 
obtained.  Individual  descriptions  of  campaign  specific  issues  are  ordered  according  to 
their  parent  issue,  as  shovm  in  Figure  5.  For  example,  the  first  definitions  presented  are 
those  under  the  Host  Nation  factor,  the  left-most  parent  issue  in  the  tier  below  Campaign 
Specific  Issues  in  the  hierarchy. 
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Campaign  Issues 


Best  Value 
Asset  Set 
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Campaign  Specific  Generic  Mission 
Goodness  Goodness 
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Ability  to 
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Inter-Service 
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1  1 

_ _ 

Lise 
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II  Requirements  || 

Asset  Mission 
Flexibility 


Availability 
of  Asset 


Enemy  vs.  USAF 
Capability 


Interference 


Munitions 
Storage  Area 


I  Fighter  Munitions  | 
j  Bomber  Munitions  I 


Tanker  Support 
Required 


Key: 


^  --  Campaign  Specific  Measures 

”!  --  Measure  of  technical  capabilities  of 
individual  aircraft  assets 
1  --  Campaign  Specific  Constraints 


FIGURE  5:  Updated  campaign  issues  Phase  II  value  hierarchy. 


HOST  NATION  ISSUES: 

Multi-National  Compatibility.  The  extent  to  which  host-nation  assets,  to  include  combat 
aircraft,  equipment,  spares,  and  support  equipment  can  be  incorporated  alongside  and  into 
USAF  combat  aircraft. 

Beddown  Location:  Fighter  Distance.  The  effectiveness  of  an  individual  fighter  asset 
within  the  asset  set  is  dependent  upon  the  distance  from  the  staging  base  to  the  target. 

This  issue  will  result  in  a  different  level  of  value  to  the  theater  commander  depending 
upon  whether  fighter  aircraft  assets  could  obtain  aerial  refueling  between  the  staging  base 
and  the  enemy  target,  and  whether  fighter  or  bomber  assets  were  being  used. 

Beddown  Location:  Bomber  Distance.  The  effectiveness  of  an  individual  bomber  asset 
within  the  asset  set  is  dependent  upon  the  distance  from  the  staging  base  to  the  target. 
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This  issue  results  in  a  different  level  of  value  to  the  theater  conunander  depending  upon 
whether  bomber  aircraft  assets  could  obtain  aerial  refueling  between  the  staging  base  and 
the  enemy  target,  and  whether  fighter  or  bomber  assets  are  being  used. 

Re-supply  Ability.  Sustained  combat  operations  from  the  staging  base  are  dependent 
upon  the  ability  of  USAF  force’s  equipment  and  materials  to  be  resupplied.  Assuming 
the  staging  base  has  been  predetermined,  the  selection  of  combat  aircraft  assets  is 
dependent  upon  individual  asset’s  resupply  requirements.  Combat  aircraft  that  require 
smaller  amounts  of  POL,  munitions,  and  other  equipment  have  a  higher  inherent  value 
than  an  aircraft  type  that  requires  more  assets. 

Allow  Assets  In.  This  issue  is  a  constraint.  A  Host-Nation  political  issue  regarding 
whether  or  not  the  specific  combat  aircraft  assets  in  a  particular  asset  set  can  be  allowed 
into  the  combat  theater  and  into  the  country  where  the  staging  base  is  located.  Either  the 
assets  in  the  particular  asset  set  can  be  allowed  into  the  theater  and  host-nation  or  they 
cannot  be  allowed  into  the  theater  and  host-nation. 

POLITICAL  ISSUES: 

International  Politics.  This  issue  is  a  constraint.  This  issue  assesses  whether 
international  political  issues,  such  as  economics,  trade  pacts,  or  host-nation  requirements 
(i.e.  the  host-nation  will  not  allow  offensive  platforms  onto  their  soil  or  over  their 
airspace),  influence  the  allowance  of  a  particular  asset  set. 

Intra-national  Politics.  This  issue  is  a  constraint.  This  issue  assesses  whether  intra¬ 
national  political  issues,  such  as  national  economics  or  congressional  debates  (i.e.  one 
state’s  congressional  representative  pushing  for  a  deployment  tasking  at  the  base  and  base 
assets  within  their  particular  district),  influence  the  allowance  of  a  particular  asset  set. 
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Inter-service  Politics.  This  issue  is  a  constraint.  This  issue  assesses  whether  inter¬ 
service  political  issues,  such  as  current  department  of  defense  policies  or  command 
leveling  (i.e.  the  desire  to  distribute  deployment  requirements  across  Army,  Navy,  Air 
Force,  and  Marine  forces),  influence  the  allowance  of  a  particular  asset  set. 

Intra-service  Politics.  This  issue  is  a  constraint.  This  issue  assesses  whether  intra¬ 
service  political  issues,  such  as  squadron  rotational  requirements  or  asset  use  desires  (i.e. 
the  desire  to  show  the  capabilities  of  the  B-2),  influence  the  allowance  of  a  particular 
asset  set. 

STAGING  BASE  ISSUES: 

Airfield  Issues:  Runway  Length.  This  issue  is  a  constraint.  The  length  of  the  runway  at 
the  pre-determined  staging  base  determines  whether  the  combat  aircraft  within  a 
particular  aircraft  asset  set  can  land  and  takeoff  at  that  staging  base.  This  issue  assesses 
whether  or  not  all  of  the  combat  aircraft  within  a  particular  aircraft  asset  set  can  land  and 
takeoff  from  the  pre-determined  staging  base. 

Airfield  Issues:  Weight  Capacity.  This  issue  is  a  constraint.  The  weight  bearing  capacity 
of  the  runway,  taxiways,  and  ramp  space  at  the  pre-determined  staging  base  determines 
whether  the  combat  aircraft  within  a  particular  aircraft  asset  set  can  conduct  operations 
from  the  pre-determined  staging  base.  This  issue  assesses  whether  or  not  all  of  the 
combat  aircraft  within  a  particular  aircraft  asset  set  can  conduct  operations  from  the  pre¬ 
determined  staging  base. 

Ramp  Space.  This  issue  is  a  constraint.  This  issue  assesses  the  capacity  at  the  pre¬ 
determined  staging  base  to  beddown  all  of  the  combat  aircraft  within  a  particular  aircraft 
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asset  set.  The  staging  base  is  either  able  or  not  able  to  beddown  all  of  the  combat  aircraft 
within  a  particular  aircraft  asset  set. 

Munitions  Storage:  Fighter  Assets.  The  Continued  fighter  operations  depend  heavily  on 
the  pre-determined  staging  base’s  ability  to  store  specific  munitions  for  fighter 
operations.  Fighter  aircraft  can  only  bring  one  load  of  munitions;  additional  sorties 
require  the  ability  of  the  staging  base  to  re-load  the  fighter. 

Munitions  Storage:  Bomber  Assets.  The  Continued  bomber  operations  depend  on  the 
pre-determined  staging  base’s  ability  to  store  specific  munitions  for  bomber  operations. 
Bomber  aircraft  can  only  bring  one  load  of  munitions;  additional  sorties  require  the 
ability  of  the  staging  base  to  re-load  the  bomber. 

Petroleum,  Oil,  and  Lubricants  (POL)  Capability.  The  staging  base’s  POL  system 
includes  storage  and  supply  from  both  military  and  host-nation  contracted  sources.  POL 
capability  for  a  particular  asset  set  must  take  into  consideration  the  per-sortie  POL 
requirements  of  each  individual  aircraft  in  the  asset  set  as  well  as  the  daily  sortie 
requirements  for  each  individual  aircraft. 

Tanker  Support  Required.  Depending  upon  the  location  of  the  staging  base  from  the 
enemy  target  and  the  fuel  requirements  per  aircraft  asset  sortie,  and  the  number  of  aircraft 
assets  conducting  sorties  at  a  particular  time,  multiple  tanker  assets  may  be  required. 
LOCATION  ISSUES: 

Topography.  This  issue  measures  the  extent  to  which  natural  and  man-made  land 
formations  around  the  pre-determined  staging  base  affect  the  ability  of  individual  aircraft 
assets  within  an  asset  set  to  take-off  and  land  at  the  staging  base. 
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Climate.  This  issue  measures  the  extent  to  which  heat,  humidity,  adverse  wind  and 
atmospheric  pressure  affect  the  ability  of  particular  aircraft  assets  to  conduct  combat 
operations. 

OPERATIONAL  USE  ISSUES: 

Availability  of  Assets.  This  issue  measures  the  extent  to  which  aircraft  assets  in  the  asset 
set  are  operationally  capable  of  being  used  to  conduct  aerospace  missions  required  by  the 
theater  commander. 

Asset  Mission  Flexibility.  This  issue  measures  the  extent  to  which  the  aircraft  within  the 
asset  set  can  perform  all  of  the  possible  aerospace  missions  defined  in  AFM  1-1  and 
AFDD  2-1 .  The  relative  worth  of  the  asset  set  is  high  if  that  asset  set  could  perform  any 
mission  required  by  the  theater  commander. 

Asset  Set  Utilization.  The  extent  to  which  aircraft  in  the  asset  set  are  used  to  perform 
aerospace  missions  as  required  by  the  theater  commander.  This  assumes  that  aircraft 
brought  into  the  theater  are  used  to  conduct  combat  sorties  and  are  not  brought  in  only  as 
a  show  of  force. 

Asset  Set  Over -Utilization.  The  extent  to  which  aircraft  in  the  asset  set  are  over-used  to 
perform  aerospace  missions  as  required  by  the  theater  commander.  This  assumes  that 
aircraft  brought  into  the  theater  are  being  used  beyond  their  operationally  sustainable 
maximum  limit. 

ENEMYISSUES: 

Enemy  Ability  and  Capabilities  versus  Asset  Set.  This  issue  measures  the  comparison  of 
enemy  assets  to  USAF  combat  aircraft  assets.  Assuming  US  intelligence  can  determine 
the  extent  of  the  enemy’s  capability,  this  issue  shows  the  relative  worth  of  the  amount  of 
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aircraft  assets  brought  into  the  pre-determined  staging  base.  One  consideration  within 
this  issue  is  the  number  of  USAF  combat  aircraft  brought  into  the  staging  base;  another 
consideration  is  the  capabilities  of  the  enemy  to  counter  the  USAF  combat  aircraft 
brought  into  the  staging  base. 

Enemy  Intelligence,  Surveillance,  and  Reconnaissance  (ISR).  This  issue  captures  the 
enemy’s  ability  to  see  or  obtain  information  on  US  activity  at  the  pre-determined  staging 
base.  Enemy  ISR  can  be  obtained  through  aerial  or  satellite  photography,  local 
sympathizers  monitoring  US  activity,  or  spies  within  the  host-nation  working  within  or 
around  the  staging  base. 

Enemy  Interference.  This  issue  captures  the  enemy’s  ability  to  interfere  with  US 
operations  at  the  pre-determined  staging  base.  Enemy  interference  is  any  action  on  the 
part  of  the  enemy  that  hampers  US  combat  operations.  Enemy  interference  could  range 
from  picketing  and  loitering  around  the  staging  base,  isolated  attempts  to  gain  access  into 
the  base  and  base  operations,  bomb  threats,  attacks  on  aircraft  taking  off  and  landing  at 
the  base,  or  attacks  on  the  base  itself. 

ON-GOING  ISSUES: 

On-Going  Requirements.  This  issue  is  a  constraint.  This  issue  assesses  whether  or  not 
enough  of  a  particular  type  of  asset  is  available  to  meet  the  number  required  in  a 
particular  asset  set. 

Once  a  list  of  campaign  specific  issues  (developed  through  CTA  interviews  with 
Headquarters  USAF,  ACC,  and  Air  University  staff  members  working  current  and  future 
deliberate  and  crisis  action  plans)  had  been  developed  into  the  value  hierarchy  shown  in 
Figure  2,  the  value  focused  decision  support  tool  could  be  developed.  This  author 
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believes  that  these  campaign  specific  issues  can  be  applied  to  any  theater  requiring  USAF 
assets  to  respond  to  a  contingency  situation.  However,  the  ranges  of  scoring  that  each 
aircraft  and  aircraft  asset  set  may  be  given  under  any  particular  campaign  specific  issue 
may  be  different  at  separate  theaters.  Therefore,  it  is  important  to  first  develop  the 
decision  support  tool  to  support  a  specific  theater.  This  decision  support  tool  was 
developed  to  foeus  on  erisis-action  planning,  through  a  value-focused  interview  with 
members  of  the  deliberate  and  crisis  action  planning  offices  at  Headquarters,  Air  Force 
Central  Command. 

PHASE  II  -  VALUE  FOCUSED  THINKING  STUDY 

The  Crisis  Action  Planning  office  at  Headquarters,  Air  Force  Central  Command 
(CENTAF),  was  contacted  to  participate  in  the  Value  Focused  Thinking  (VFT)  interview 
to  develop  quantitative  ranges  of  scores  for  each  of  the  factors  presented  in  Figure  5. 
During  the  visit,  three  other  CENTAF  campaign  planners  were  also  interviewed.  Two 
separate  interviews  were  condueted. 

At  the  beginning  of  the  interview,  the  interviewees  were  given  a  brief  introduction 
to  Value-Focused  Thinking.  The  new  car  purchase  example  in  Appendix  E  was 
presented  and  used  to  familiarize  the  interviewees  with  the  goals  of  the  interview  and  to 
gain  an  understanding  on  how  the  interview  would  be  conducted.  Once  the  interviewees 
were  confident  that  they  understood  the  process,  the  value  elicitation  began. 

The  value  elicitation  interview  was  setup  as  discussed  in  the  Phase  II  methodology 
section  of  this  paper.  The  interview  was  conducted  by  showing  a  single  campaign 
specific  issue  graph  to  the  interviewee  and  eliciting  how  the  graphical  function, 
converting  the  specific  issue’s  measurement  to  value,  would  look.  First,  the  interviewee 


73 


was  asked  if  the  X-axis  measurement  made  sense  for  the  specific  issue,  and  if  some 
different  scale  or  measurement  would  be  more  effective.  In  several  instances,  the  pre¬ 
determined  measures  were  adjusted  based  upon  the  interviewee’s  knowledge  and 
understanding  of  campaign  planning. 

Some  of  the  campaign  specific  issue  measurements  were  changed  during  the 
group  interview.  For  instance,  the  measurement  for  the  Ability  to  Resupply  variable  was 
adjusted  to  better  reflect  the  commander’s  views  and  requirements.  This  measurement 
had  originally  been  developed  to  measure  the  distance  of  the  pre-determined  staging  base 
to  the  supply  Point  of  Debarkation  (POD).  This  measurement  was  changed  to  a  ratio  of 
consumption  to  resupply.  The  interviewee  group,  comparing  possible  alterations  to  the 
measurement,  stated  that  the  ratio  function  was  better  suited  to  assisting  campaign 
plaimers  in  selecting  the  best  set  of  aircraft  assets  to  respond  to  a  theater  crisis.  The 
Tanker  Support  Required,  Enemy,  and  Political  issues  were  also  adjusted  during  the 
interview  as  to  how  they  were  measured  in  order  to  better  reflect  the  views  and 
requirements  of  the  theater  commander. 

Once  the  graphical  functions  were  obtained  from  the  Headquarters,  USAF  Central 
Command  campaign  planning  staff,  the  functions  were  incorporated  into  an  Excel 
spreadsheet.  The  eampaign  specific  planning  issues  were  incorporated  into  the  decision 
support  tool  using  graphical  function  equations  developed  by  Kirkwood  [Kirkwood, 

1997;  pg  75-85]. 

During  the  coding  of  these  funetions,  it  was  noted  that  5  value  functions,  elicited 
from  the  two  separate  group  interviews,  were  dissimilar.  The  issues  in  question  were 
Climate,  POL,  Asset-Mission  Flexibility,  MSA  Fighter  Support,  and  MSA  Bomber 
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Support.  An  individual  Excel  worksheet  for  each  of  the  separate  issues  was  developed, 
presenting  both  of  the  dissimilar  graphical  functions.  These  worksheets  were  then  e- 
mailed  to  the  interviewees  individually,  so  that  they  could  compare  the  opposing 
graphical  functions  of  each  issue  in  question  and  provide  feedback  on  which  graphical 
representation  was  better,  or  if  there  was  another  representation  that  better  suited  the 
measurement  of  these  individual  issues.  For  each  issue  in  question,  the  campaign 
plarmers  at  Headquarters,  USAF  Central  Command  individually  concurred  with  a  single 
graphical  representation.  The  agreed  upon  graphical  representation  for  each  issue  has 
been  incorporated  into  the  decision  support  tool.  The  graphical  representation  for  each 
individual  campaign  specific  issue,  along  with  the  definitions  and  quantitative 
measurement  values,  are  presented  collectively  as  Appendix  I. 

The  final  step  in  developing  the  Value  Focused  decision  support  tool  was  to 
obtain  relative  weights  for  each  of  the  campaign  specific  issues.  Even  more  than  the 
measurements  of  the  issues  themselves,  the  weights  of  these  campaign  specific  issues 
may  be  very  dependent  upon  the  location  or  theater  in  which  the  crisis  is  taking  place.  A 
Delphi  study  was  applied  in  Phase  III  of  this  research  to  obtain  consensus  of  the  value 
hierarchy  as  developed  from  all  subject  matter  experts  queried  in  this  research.  The 
Delphi  study  was  also  used  to  obtain  Southwest  Asia  Theater  specific  weights  for  each  of 
the  campaign  specific  issues. 

PHASE  III  -  DELPHI  STUDY 

Ten  of  the  25  SMEs  returned  a  completed  first  questionnaire,  for  a  response  rate 
of  40  percent.  Five  other  SMEs  stated  that  they  could  not  complete  the  survey  due  to 
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business  requirements,  but  would  try  to  complete  the  second  questioimaire  when  it  was 
received.  Overall,  the  reply  rate  of  the  first  round  was  60  percent.  Of  the  three  purposes 
for  this  first  round  of  the  Delphi  study,  this  questionnaire  satisfied  the  first  of  its  three 
purposes.  Presented  as  Appendix  K,  there  were  a  number  of  constructive  comments 
regarding  the  configuration  of  campaign  specific  issues  within  the  value  hierarchy.  As 
expected  however,  this  first  questionnaire  did  not  provide  any  clear  picture  as  to  a 
consensus  of  the  relative  ranking  of  individual  campaign  specific  issues.  The  individual 
rankings  of  campaign  specific  issues  are  presented  in  Table  3.  Campaign  specific  issues 
in  italic  font  represent  campaign  specific  constraints;  they  are  either  yej'  or  no  alternatives 
and  do  not  possess  value  functions  as  do  the  campaign  specific  issues  presented  in 
Appendix  I.  Note  that  a  general  trend  among  the  responding  participants  can  he  seen 
with  respect  to  the  issues  believed  to  be  most-potentially  influential  and  the  least- 
potentially  influential.  Allow  Assets  In  and  Intra-National  Politics  were  generally  ranked 
as  potentially  having  a  large  influence  on  the  selection  of  combat  aircraft.  Conversely, 
Multi-National  Compatibility,  Intra-Service  and  Inter-Service  Politics,  Topography, 
Climate,  and  On-Going  Requirements  were  generally  ranked  as  potentially  having  a  small 
influence  on  the  selection  of  combat  aircraft.  Based  upon  the  results  of  this  first 
questiormaire,  the  second  questionnaire  was  built.  The  purpose  of  the  second 
questiormaire  was  to  elicit  discussion  on  the  comments  obtained  from  the  first 
questiormaire,  and  to  present  the  rankings  obtained  from  the  first  questiormaire  in  an 
effort  to  obtain  a  more  discernable  trend  to  the  individual  rankings. 

The  second  questiormaire  in  this  first  round  was  developed  similar  to  the  first 
questiormaire.  The  two  basic  questions,  as  proposed  in  the  first  round,  were  asked.  For 
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Question  A  in  this  round,  the  comments  obtained  from  the  first  round  were  summarized 
and  provided.  Each  survey  participant  was  asked  to  provide  feedback  and  discussion  on 
the  comments,  whether  positive  or  negative.  The  experts  were  again  asked  about  the 
ranking  of  the  campaign  specific  issues  based  upon  their  knowledge  and  expertise  for 
Question  B. 


1  TABLE  3:  Delphi  Study  Questionnaire  1  Ranking  Results 

INDIVIDUAL  RESPONSES  | 

ACC 

> 

O 

O 

ACC 

ACC 

CAF 

HQ 

HQ 

HQ 

AU 

AU 

CAMPAIGN  SPECIFIC  ISSUES 

4 

5 

6 

7 

2 

2 

3 

4 

1 

2 

Multi-National  Capability 

15 

21 

19 

22 

23 

22 

22 

24 

21 

18 

Beddown  Location:  Fighter  Distance 

16 

10 

11 

5 

4 

10 

8 

2 

8 

7 

Beddown  Location:  Bomber  Distance 

21 

10 

12 

6 

11 

17 

8 

3 

19 

8 

Ability  to  Resupply 

11 

17 

10 

17 

_ 

15 

18 

19 

23 

6 

9 

Allow  Asssets  In 

1 

18 

1 

4 

1 

2 

3 

1 

1 

10 

Inter-National  Politics 

2 

1 

2 

2 

17 

1 

9 

10 

3 

5 

Intra-National  Politics 

3 

2 

13 

3 

18 

7 

9 

11 

2 

23 

Inter-Service  Politics 

25 

23 

18 

23 

19 

6 

20 

22 

22 

21 

Intra-Service  Politics 

26 

24 

26 

24 

20 

22 

21 

21 

23 

22 

Airfield:  Runway  Length 

13 

11 

5 

7 

6 

8 

4 

12 

17 

4 

Airfield:  Weight  Capacity 

14 

12 

6 

8 

14 

9 

4 

13 

18 

5 

Ramp  Space 

12 

13 

3 

9 

3 

22 

5 

14 

11 

6 

Munitions  Storage  Area:  Fighter  Munitions 

9 

14 

7 

10 

7 

12 

6 

15 

9 

14 

Munitions  Storage  Area:  Bomber  Munitions 

10 

14 

8 

11 

12 

21 

6 

16 

20 

15 

Petroleum,  Oil,  and  Lubricants 

7 

15 

4 

12 

5 

;  13 

7 

17 

10 

5 

Tanker  Support  Required 

8 

16 

14 

13 

13 

11 

16 

4: 

7 

12 

Topography 

23 

20 

16 

26 

24 

19 

17 

18 

5 

16 

Climate 

22 

19 

17 

25 

25 

20 

18 

19 

4 

17 

Asset  Mission  Flexibility 

18 

4 

23 

21 

2 

14 

1 

25 

12 

2 

Assset  Set  Utilization 

19 

5 

24 

19 

16 

15 

2 

26 

24 

13 

Asset  Set  Over-Utilization 

17 

6 

25 

20 

21 

16 

11 

20 

25 

18 

Availability  of  Asset 

20 

3 

9 

18 

22 

5 

10 

5 

16 

11 

Enemy  Vs.  USAF  Capability 

5 

7 

21 

14 

8 

3 

12 

6 

14 

19 

Enemy  ISR 

6 

8 

15 

15 

9 

4 

15 

9 

13 

r 

Enemy  Interference 

4 

9 

20 

16 

10 

4 

14 

8 

15 

20 

On-Going  Requirements 

24 

22 

22 

1 

26 

23 

13 

7 

26 

3 

However,  in  this  round,  the  rankings  obtained  from  the  first  questionnaire  were  provided 
to  the  experts.  The  previous  questionnaire’s  findings  were  presented  to  the  study  group 
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in  this  second  questionnaire  to  provide  feedback  as  to  the  range  of  rankings  obtained 
from  the  first  questionnaire.  The  findings  were  also  included  to  help  those  participants 
who  did  not  complete  the  first  questionnaire  to  both  understand  what  is  being  asked  of 
those  participants  and  to  provide  them  with  a  tool  in  which  they  could  submit  their 
opinions. 

As  there  were  only  four  responses  to  the  second  questionnaire  at  the  time 
responses  were  requested,  the  questionnaire  was  rewritten  and  resent.  A  more  concise, 
“Yes”  or  “No”  question  was  asked  of  the  survey  pool  members  who  did  not  respond  to 
the  second  questionnaire.  In  this  draft,  the  campaign  specific  issues  were  separated  into 
three  categories:  most  potentially  important,  potentially  important,  and  least  potentially 
important  to  the  selection  of  combat  aircraft  to  the  SWA  theater.  Group  number  1 
corresponds  to  the  most-potentially  important  issues.  Group  2  corresponds  to  important 
issues,  and  Group  3  corresponds  to  least  potentially  important  issues.  If  the  participants 
did  not  agree  with  the  groupings  of  the  issues,  they  were  asked  to  re-group  the  issues 
according  to  their  group  number.  The  short  version  of  the  second  questionnaire  received 
five  additional  responses,  bringing  the  response  rate  of  the  second  questionnaire  up  to  9 
out  of  25,  or  36  percent.  The  initial  grouping  of  Campaign  Specific  Issues  for  this 
questionnaire,  accompanied  by  the  answers  provided  by  the  study  group  respondents  is 
presented  as  Table  4. 

Based  upon  comments  made  during  the  first  and  second  questionnaires  of  the  first 
phase  of  the  Delphi  study,  the  Campaign  Specific  Issue  value  hierarchy  was  updated. 

The  updated  value  hierarchy  is  shown  as  Figure  6. 
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TABLE  4:  Delphi  Study 

Initial 

CAMPAIGN  SPECIFIC  given 

_ ISSUES _ for  use 

Allow  Assets  In? _ 1 

Inter-National  Politics _ 1 

Tanker  Support  Required _ 1 

Asset-Mission  Flexibility _ 1 

Beddown  Location:  Fighter  Dist _ 2 

Beddown  Location:  Bomber  Dist  2 

Intra-National  Politics _ 2 

Inter-Service  Politics _ 2 

Airfield:  Runway  Length _ 2 

Airfield:  Weight  Capacity _ 2 

Ramp  Space _ 2 

MSA:  Fighter  munitions _ 2 

MSA:  Bomber  munitions  2 

POL  2 

Topography _ 2 

Climate _ 2 

Asset  Set  Utilization _ 2 

Asset  Set  Over-Utilization _ 2 

Availability  of  Asset _ 2 

USAF  Vs.  Enemy  Capability _ 2 

Enemy  ISR _ 2 

Enemy  Interference _ 2 

On-Going  Requirements _ 2 

Ability  to  Resupply _ 3 

Multi-National  Compatibility _ 3 

Intra-Service  Politics  3 


Questionnaire  2  Group  Ranking  Results 

RESPONSES 

ACC  I  ACC  I  ACC  I  ACC  I  CAF  I  HQ  I  HQ 


HQ  AU 
5  2 
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Campaign  Issues 


Best  Value 
Asset  Set 
Per  Phase 


I  Weight  Capacity  | 


Runway  Weight 


Runway  Length  I 


POL 

Tanker 

Support 

J 

Bomber 

Specific 

Fighter 

Specific 

1 

Bomber  Distanc* 


I  Bomber  Munitions 


Topography  i 

_ 

i 

Operating 

Host-Nation 

1  Politics  |i 

Operational 

Enemy 

On-Going 

Location 

- 1 - 

1 

1.  II 

Use 

1 

L - 

Requirements 

Multi-National 

Compatibility 


Ability  to 
Resupply 


Allow  Assets 


n 


Inter-National  M 


Intra-National 


Inter-Service 


Asset  Mission 
Flexibility 


US  vs  Enemy 
Capability 


I  Fighter  Distance  | 
I  Fighter  Munitions  I 


Key: 


^  —  Campnign  Specific  Measures 


_J  —  Measure  of  technical  capabilities  of  individual  aircraft  assets 
T  —  Campaign  Specific  Constraints 


FIGURE  6:  Delphi  Study  Updated  campaign  issues  value  hierarchy. 


The  first  change  included  the  movement  of  the  fighter  and  bomber  distance  issues 
to  the  staging  base  parent  issue.  Secondly,  the  parent  issue  Staging  Base  was  re-named 
Operating  Location,  because  this  term  more  accurately  describes  the  location  at  which 
fighters  and  bombers  conduct  missions  from.  The  term  Staging  Base  may  refer  to 
locations  where  supplies  and  equipment  are  brought  into  theater  from  the  continental  US 
(CONUS),  before  they  are  shipped  to  the  operating  location.  Additionally,  the  Operating 
Location  for  certain  aircraft  platforms,  primarily  bomber  assets,  may  be  located  in 
CONUS.  Also,  fighter  and  bomber  distance  locations  is  not  a  Host-Nation  issue.  A  third 
change  was  the  addition  of  a  Runway  Weight  Capacity  constraint  to  the  Airfield 
constraint  issue.  The  weight  capacity  of  the  operating  location  runway  may  be  an 
important  issue  to  consider  when  bringing  in  certain  aircraft  assets.  A  final  change  was 
to  regroup  the  distance  and  Munitions  Storage  Area  (MSA)  issues  for  both  fighter  and 
bomber  assets.  Fighter  Specific  Issues,  which  includes  both  fighter  distance  and  MSA: 
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Fighter  munitions,  and  Bomber  Specific  Issues,  which  includes  both  bomber  distance  and 
MSA:  Bomber  munitions,  is  a  better  representation  of  these  issue. 

1 5  of  the  25  Delphi  study  pool  members  were  given  the  third  e-mail 
questionnaire.  The  individuals  given  the  e-mail  questionnaire  were  those  that  were  not 
interviewed  directly  as  a  part  of  the  second  weighting  elicitation  method.  10  members  of 
the  Delphi  study  pool  were  contacted  and  interviewed. 

Seven  individual  sets  of  weights  were  obtained  through  the  interview  and  e-mail 
questionnaires.  One  set  of  parent-issue  weights  was  obtained  through  an  interview  at 
HQ/USAF,  for  a  total  response  rate  of  32  percent.  The  individual  sets  of  weights  were 
transformed  and  entered  into  the  decision  support  tool  as  described.  For  each  of  these 
individual  sets  of  weights,  final  values  for  each  asset  set  were  then  calculated.  The  top 
ten  highest  final  value  asset  sets  were  then  compared  to  see  how  the  individual  sets  of 
weights  impacted  the  model  output.  Based  upon  this  analysis,  it  was  determined  that 
each  of  the  individual  sets  of  weights  gave  similar,  but  not  equal  rank  ordering  of  asset 
sets.  It  was  therefore  determined  that  the  individual  sets  of  weights  should  be  combined. 

An  average  adjusted  weight  for  each  issue  was  developed  using  individual  sets  of 
weights  that  were  adjusted  based  upon  their  organization.  Because  the  decision  support 
tool  has  been  developed  specifically  for  the  SWA  Theater,  the  set  of  weights  obtained 
from  CENTAF  was  quadrupled.  Sets  of  weights  obtained  from  ACC  were  doubled,  and 
sets  of  weights  obtained  from  Air  University  and  HQ/USAF  were  not  adjusted.  Weights 
from  ACC  were  doubled  due  to  the  current  focus  on  the  SWA  Theater,  but  ACC  is  not  as 
dedicated  to  the  SWA  Theater  as  is  CENTAF.  Air  University  and  HQ/USAF  were  not 
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adjusted  based  on  the  belief  that  their  campaign  planning  focus  is  spread  throughout  all 
possible  theaters,  not  just  the  SWA  Theater.  The  adjusted  weights  were  then  summed 
and  then  divided  by  the  total  adjusted  number  of  sets  of  weights.  One  set  of  weights  was 
obtained  from  CENTAF,  three  sets  of  weights  were  obtained  from  ACC,  and  one  set  each 
of  weights  was  obtained  from  Air  University  and  HQ/USAF.  These  averaged,  adjusted 
weights  in  the  model,  along  with  the  range  of  weights  calculated  for  each  campaign 
specific  issue,  is  presented  in  Table  5.  A  discussion  on  the  sensitivity  analysis  of  these 
weights  as  they  may  affect  the  final  value  of  the  alternative  asset  sets  is  presented  in  the 
next  section  of  this  chapter. 

The  second  questionnaire  in  this  phase  of  the  Delphi  study  was  used  to  confirm 
the  adjusted  weights  for  each  individual  campaign  specific  issue.  This  questionnaire  was 
also  used  to  confirm  changes  made  to  the  campaign  specific  issue  value  hierarchy  from 
the  comments  made  in  the  first  phase  of  the  Delphi  study.  As  with  the  first  two 
questionnaires,  the  survey  participants  were  e-mailed  the  questionnaire  and  asked  to  reply 
by  e-mail  with  their  responses.  For  this  questionnaire,  three  individual  survey 
participants  from  ACC  were  not  included.  These  individuals  were  not  included  because 
of  a  disinterest  in  this  research,  and  it  was  felt  that  the  inclusion  of  these  individuals 
within  this  final  round  would  not  be  beneficial  to  any  of  the  parties  involved. 
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TABLE  5:  Delphi  Study  Identified  Global  Weights 

Campaign  Specific  Issue 

Global  Weight  Identified  | 

Low 

Adjusted 

Average 

High 

HOST  NATION 

Multi-National  Compatibility 

0.0039 

0.068 

0.1392 

Ability  to  Resupply 

0.174 

0.3230 

OPERATING  LOCATION 

Beddown  Location:  Fighter  Dist 

0.0114 

0.024 

0.0360 

Beddown  Location:  Bomber  Dist 

0.0240 

0.099 

0.1480 

POL 

0.0209 

0.058 

0.1200 

MSA:  Fighter  munitions 

0.0133 

0.037 

0.0480 

MSA:  Bomber  munitions 

0.0133 

0.032 

0.0480 

Tanker  Support  Required 

0.0152 

0.060 

0.2820 

Climate 

0.0038 

0.015 

0.0240 

Topography 

0.0019 

0.018 

0.0320 

OPERATIONAL  USE 

Asset-Mission  Flexibility 

0.0336 

0.067 

0.0825 

Availability  of  Asset 

0.0272 

0.064 

0.1045 

Asset  Set  Utilization 

0.0323 

0.051 

0.0992 

ENEMY  1 

USAF  Vs.  Enemy  Capability 

0.0660 

0.118 

0.1440 

Enemy  ISR 

0.0012 

0.046 

0.1023 

Enemy  Interference 

0.0360 

0.071 

0.1110 

Eight  of  the  remaining  22  survey  participants  replied  to  the  fourth  questioimaire, 
for  a  response  rate  of  36  percent.  Based  upon  the  responses  provided,  no  changes  were 
made  to  the  campaign  specific  issue  value  hierarchy  or  the  individual  weights  of  the 
issues  within  the  hierarchy. 


SENSITIVITY  ANALYSIS  OF  CAMPAIGN-SPECIFIC  ISSUE  WEIGHTS 

Because  no  consensus  was  reached  for  the  individual  global  issue  weights,  a 
sensitivity  analysis  was  conducted  on  the  parent  campaign-specific  issue  weights.  The 
purpose  of  this  analysis  is  to  identify  how  changes  in  the  weight  of  each  of  the  parent 
issues  affect  the  relative  value  of  an  individual  asset  set.  While  the  weights  may  change, 
simply  changing  the  weights  does  not  change  the  value  of  every  alternative.  It  is 
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important  to  note  that  all  though  the  data  is  notional,  this  data  will  not  affect  changes  in 
the  ranking  of  individual  asset  sets  in  the  same  manner  as  will  the  real-world  developed 
issue  weightings. 

Using  the  developed  adjusted  average  weights  presented  in  Table  5,  a  sensitivity 
analysis  was  separately  conducted  on  each  of  the  four  parent  issues;  Host  Nation, 
Operational  Location,  Operations,  and  Enemy.  The  sensitivity  analyses  were  conducted 
by  swinging  the  global  weights  of  the  tested  parent  issue  from  10-percent  to  90-percent, 
maintaining  the  same  ratio  for  the  other  three  weights.  At  some  point  in  each  of  the 
analyses,  the  asset  set  identified  as  having  the  highest  total  value  (best  value)  changed. 
Figure  7  presents  the  highest  total  value  asset  sets  along  the  range  of  weighting  of  the 
Host  Nation  issue.  Figure  8  presents  the  highest  total  value  asset  sets  along  the  range  of 
weighting  of  the  Operational  Location  issue.  Figure  9  presents  the  highest  total  value 
asset  sets  along  the  range  of  weighting  of  the  Operations  issue.  Figure  10  presents  the 
highest  total  value  asset  sets  along  the  range  of  weighting  of  the  Enemy  issue. 
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FIGURE  7;  Sensitivity  analysis  on  Host  Nation  global  weight 
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As  shown  in  Figure  7,  Asset  set  #31  has  the  highest  final  value  at  low  values  of 
the  Host  Nation  global  weight.  Correspondingly,  when  The  Host  Nation  weight  is  low 
and  the  Operation  Location  weight  is  high,  Asset  set  #3 1  has  the  top  rank.  This  indicates 
that  Asset  set  #3 1  is  generally  a  robust  asset  set  in  terms  of  value.  As  the  Host  Nation 
global  weight  increases,  we  can  see  that  Asset  set  #1  increases  in  total  value,  becoming 
the  top- value  asset  set.  This  is  evident  in  that  Asset  set  #1  has  a  top-score  value  (1 .0)  for 
both  Multi-National  Compatibility  and  Resupply  Ability  issues,  which  drives  the  total 
value  when  Host  Nation  is  the  most  importantly  weighted  parent  issue.  Looking  over  the 
range  of  global  weights  obtained  from  the  Delphi  study,  Asset  sets  #31  and  #17  are  the 
top-value  alternatives.  Comparing  the  individual  responses  to  the  range  in  Figure  7, 
weights  corresponding  to  the  selection  of  Asset  set  #31  were  generally  seen  by  subject 
matter  experts  (SMEs)  from  HQ/USAF,  ACC,  and  AU.  Weights  corresponding  to  the 
selection  of  Asset  set  #17  were  seen  by  SMEs  from  CENTAF. 
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Sensitivity  Analysis:  Location  Weight 
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FIGURE  8:  Sensitivity  analysis  of  Operation  Location  global  weight 


As  shown  in  Figure  8,  Asset  sets  #17  and  #36  are  very  similar  in  total  value 
throughout  the  range  of  possible  Operation  Location  issue  weight.  The  main  difference 
between  Asset  sets  #17  and  #3 1  are  with  respect  to  their  value  contributed  from  the 
Beddown  Location:  Bomber  Assets  and  POL  issues.  These  two  issues  do  have  a 
relatively  high  individual  weight,  compared  to  the  other  campaign-specific  issues. 
Throughout  the  range  of  global  weights  obtained  from  the  Delphi  study,  Asset  sets  #3 1 
and  #17  remain  the  top- value  alternatives.  Comparing  the  individual  responses  to  the 
range  in  Figure  8,  global  weights  corresponding  to  the  selection  of  Asset  set  #17  were 
seen  by  CENTAF.  Global  weights  corresponding  to  the  selection  of  Asset  set  #31  were 
generally  seen  by  subject  matter  experts  (SMEs)  from  HQ/USAF  and  AU.  ACC  SMEs 
provided  weights  that  corresponded  to  both  Asset  sets. 
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Sensitivity  Analysis:  Operations 
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FIGURE  9:  Sensitivity  analysis  on  Operations  global  weight 


As  shown  in  Figure  9,  Asset  sets  #21  and  #36  generally  have  the  same 
characteristics  as  the  Operations  parent  issue  global  weight  is  swung  from  a  low  value  to 
a  high  value.  At  a  high  weight  for  the  Operations  issues,  relative  to  the  other  three  parent 
issues,  Asset  set  #21  has  the  highest  total  value.  This  is  evident  due  to  the  individual 
high  values  given  under  the  Availability  and  Flexibility  issues,  both  of  which  have 
relatively  high  individual  weights.  At  a  low  global  weight  for  the  Operations  issues, 
relative  to  the  other  three  parent  issues.  Asset  set  #3 1  has  the  highest  total  value.  At  the 
extreme  low  global  weight  of  0.02,  Asset  set  #9  becomes  the  set  with  the  highest  total 
value.  Looking  over  the  range  of  global  weights  obtained  from  the  Delphi  study.  Asset 
sets  #31  and  #36  are  the  top- value  alternatives.  Comparing  the  individual  responses  to 
the  range  in  Figure  9,  global  weights  corresponding  to  the  selection  of  Asset  set  #3 1  were 
seen  by  AU.  Global  weights  corresponding  to  the  selection  of  both  Asset  set  #31  and  #36 
were  generally  seen  by  subject  matter  experts  (SMEs)  from  HQ/US AF,  ACC,  and 
CENTAF,  depending  upon  the  SME  interviewed. 
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As  shown  in  Figure  10,  the  Enemy  parent  issue  is  very  sensitive  to  changes  in 
global  weight  as  compared  to  the  other  three  parent  issues.  The  highest  value  asset  set 
changes  4  times  over  the  range  of  Enemy  parent  issue  global  weight.  But  for  the  range  of 
global  weights  obtained,  there  is  no  change  in  the  order  of  these  asset  sets.  This  implies 
that  the  subject  matter  experts  seem  to  be  in  agreement  about  the  importance  of  Enemy 
issues.  At  a  high  value  for  Enemy  global  weight,  Asset  set  #7  has  the  highest  total  value. 
This  is  evident  due  to  the  top-score  value  (1.0)  for  the  Enemy  Interference  issue,  and 
relatively  high  value  for  the  Enemy  ISR  issue.  While  Asset  sets  #36  actually  has  a  higher 
individual  value  for  the  Enemy  ISR  issue,  the  higher  individual  values  for  the  non-Enemy 
issues  give  Asset  set  #7  a  higher  total  value.  This  is  evident  when  the  Enemy  parent  issue 
global  weight  is  smaller,  where  Asset  set  #7  has  a  lower  total  value  than  the  other  three 
contending  best- value  asset  sets  shown.  Looking  over  the  range  of  global  weights 
obtained  from  the  Delphi  study.  Asset  sets  #3 1  and  #36  are  the  top-value  alternatives. 
Comparing  the  individual  responses,  weights  corresponding  to  the  selection  of  Asset  set 
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#3 1  were  seen  by  SME  from  each  of  the  four  agencies.  One  set  of  weights  from  an  ACC 
SME  corresponded  to  the  selection  of  Asset  set  #36,  showing  that  there  is  some 
disagreement  between  ACC  campaign  planners. 

DEVELOPED  DECISION-SUPPORT  TOOL 

Through  the  analysis  of  results  from  each  phase  of  the  research,  a  Value  Focused 
Thinking  (VFT)  Decision  Support  Tool  to  assist  campaign  planners  in  selecting  a  better 
set  of  combat  aircraft  to  deploy  in  response  to  a  Southwest  Asian  theater  crisis  action  has 
been  developed.  The  campaign  specific  issue  decision  support  tool  was  developed  using 
the  M.T.  Winthrop,  Excel-based  VFT  analysis  spreadsheet  [Winthrop,  1999]. 

Within  the  decision  support  tool,  the  Campaign  Specific  Issue  value  hierarchy  is 
presented.  Hyperlinks  on  each  campaign  specific  issue  connects  that  issue  to  a 
spreadsheet  that  describes  the  issue  and  provides  the  mathematical  connection  between 
the  range  of  measurement  of  the  specific  issue  and  value. 

The  range  of  measurements  and  correlating  values  were  defined  and  calculated 
through  the  VFT  interview  with  CENTAF.  These  measurement-to-value  correlations  are 
used  in  the  mathematical  analysis  of  alternative  sets  of  assets.  Descriptions  for  the 
Campaign  Specific  constraints  are  also  provided.  An  individual  worksheet  is  dedicated 
to  each  single  Campaign  Specific  Issue  and  Constraint. 

Following  these  worksheets,  the  calculation  of  relative  and  global  weights  is 
presented.  This  series  of  worksheets  is  used  to  provide  the  weighting  information 
obtained  from  the  Delphi  study  questionnaire  and  interviews,  as  well  as  the  calculations 
to  determine  the  individual  Campaign  Specific  Issue  weightings  used  in  the  mathematical 
analysis  of  alternative  sets  of  assets. 
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Following  the  weight  spreadsheets,  imported  information  on  the  alternative  sets  of 
assets  is  presented.  This  information,  based  on  Major  Swartz’s  Asset  Set  Pilot  Problem, 
provides  information  specific  to  the  mathematical  analysis  of  the  asset  set  [Swartz,  1999]. 
This  information,  coupled  with  the  mathematical  value  data  and  global  weight  data,  is 
used  to  develop  the  calculation  worksheet. 

The  calculation  worksheet  is  used  to  determine  the  relative  value,  from  1  to  0,  of 
each  alternative  set  of  assets,  based  upon  the  analysis  of  the  campaign  specific  issues.  An 
asset  set  with  a  value  of  1  would  provide  the  theater  commander  with  the  best  total  value, 
based  upon  the  analysis  of  the  campaign  specific  issues.  Conversely,  an  asset  set  with  a 
value  of  0  would  either  be  determined  to  be  infeasible  based  upon  the  analysis  of  the 
Campaign  Specific  Constraints. 

The  worksheets  that  follow  the  calculation  worksheet  provide  the  campaign 
planner  with  a  graphical  representation  of  the  value  of  the  alternative  asset  sets  based 
upon  the  Campaign  Specific  Issues.  Looking  at  the  absolute  capabilities  of  the  combat 
aircraft  comprising  an  asset  set,  each  unique  set  of  assets  is  equally  capable  of  completing 
the  missions  required  by  the  Theater  Commander.  The  value  of  an  alternative  asset  set  is 
compared  to  the  amount  of  lift  required  to  deploy  that  set  of  assets  to  the  SWA  Theater. 
Sets  of  assets  with  a  higher  value  compared  to  the  amount  of  lift  required  to  deploy  them 
are  considered  better  than  those  sets  that  have  a  lower  value  with  the  same  amount  of  lift 
required.  Likewise,  sets  of  assets  with  a  lower  amount  of  lift  required  to  deploy  that  set 
is  considered  better  than  those  sets  that  require  additional  lift  for  the  same  value. 

The  Campaign  Specific  Issue  Decision  Support  Tool  was  run  using  the  50 
notional  asset  sets  identified  through  the  AFIT/ALP  Pilot  Problem.  For  each  asset  set. 
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notional  data  was  for  all  campaign  specific  issues  was  imported  into  the  spreadsheet. 
Applying  this  notional  information  to  the  campaign  specific  value  functions,  the  50  asset 
sets  were  analyzed. 


FIGURE  1 1 :  Value  of  AFIT/ALP  Pilot  Problem  Asset  Sets  vs.  Lift  required 


Once  the  relative  value  of  each  asset  set  was  determined,  the  asset  sets  were 
compared  against  the  lift  (notional)  required  to  deploy  that  particular  asset  set.  A 
graphical  analysis  of  the  relative  value  of  notional  AFIT/ALP  Pilot  Problem  asset  set 
versus  the  notional  lift  required  to  deploy  the  asset  set  is  shown  as  Figure  11.  Note  that, 
based  on  this  notional  example.  Asset  set  #17  labeled  on  Figure  1 1  has  a  very  high  value 
(relative  to  the  other  asset  sets)  while  not  requiring  as  much  lift  as  the  other  asset  sets 
with  equivalent  value  (Asset  sets  #3 1  and  #36).  Campaign  Planners  can  also  use  this  to 
single  out  a  group  of  high  value  asset  sets  for  further  study  or  presentation  as  options  to 
the  Joint  Air  Component  Commander  and/or  Joint  Forces  Commander. 
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The  Campaign  Specific  Issue  Decision  Support  Tool  will  be  integrated  into  the 
AFIT/ALP  Asset  Selection  tool.  Actual  aircraft  specific  information,  required  within  the 
decision  support  tool,  will  be  provided  through  the  Advanced  Logistics  Project  database 
system. 

SUMMARY 

This  chapter  has  detailed  the  results  of  each  phase  of  the  research,  leading  to  the 
development  and  testing  of  a  Campaign  Specific  Issue  Decision  Support  Tool.  A 
Campaign  Specific  Issue  value  hierarchy  was  developed  through  review  of  current  Air 
Force  and  Joint  Publications  and  revised  through  several  rounds  of  interviews  and 
questionnaires.  With  respect  to  a  Southwest  Asian  (SWA)  Theater,  Air  Expeditionary 
Force  (AEF)  deployment,  ranges  of  measurement  were  identified  for  each  of  the 
Campaign  Specific  Issues  within  the  value  hierarchy.  Relative  weights  of  Campaign 
Specific  Issues  were  determined,  where  the  purpose  of  this  AEF  deployment  to  the  SWA 
Theater  was  to  conduct  combat  operations  in  a  decisive  enemy  halt  scenario.  A  decision 
support  tool  to  study  these  Campaign  Specific  Issues  was  then  developed  and  tested  using 
AFIT/ALP  Pilot  Problem  asset  sets. 

The  next  and  final  chapter  of  this  document  will  discuss  the  conclusions, 
successes,  and  limitations  of  this  research.  Possible  follow-on  research  will  also  be 
suggested. 
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V.  CONCLUSION 


INTRODUCTION 

In  Chapter  1,  it  was  stated  that  the  Advanced  Logistics  Project  (ALP),  if  proven 
successful,  would  provide  a  methodology  to  quickly  develop  a  detailed  deployment  plan, 
whereby  campaign  planners  will  have  the  opportunity  to  compare  multiple  deployment 
scenarios  and  schedules.  Given  this  advance  in  campaign  planning,  how  would  campaign 
planners  choose  between  similar  deployment  plans?  As  assessment  of  competing  sets  of 
combat  aircraft  assets  could  be  based  upon  campaign  specific  or  theater  specific  issues 
that  may  not  be  addressed  by  an  aircraft’s  absolute  capability,  but  would  be  nonetheless 
important  to  the  satisfactory  completion  of  the  operational  missions  required  by  that  set 
of  assets.  Coupled  with  an  assessment  of  the  absolute  capabilities  of  the  combat  aircraft 
comprising  a  particular  set  of  assets,  campaign  planners  can  be  provided  a  tool  to  study 
both  the  absolute  and  contextual  suitability  of  an  individual  set  of  assets  to  a  specific 
theater  crisis. 

The  goals  of  this  research  were  to  assess  the  contribution  of  these  contextual, 
campaign  specific  issues  that  could  affect  the  operational  capabilities  or  mission 
effectiveness  of  the  combat  aircraft  within  the  particular  set  of  assets,  and  to  develop  a 
decision  support  tool  to  measure  these  issues.  To  accomplish  these  goals,  four  questions 
were  answered: 

1 .  What  factors  other  than  absolute  aircraft  capabilities  should  campaign  planners 
consider  when  selecting  a  specific  set  of  combat  aircraft  to  counter  a  theater 
threat? 

2.  How  are  these  factors  quantified? 
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3.  What  relationships  link  these  factors  to  each  other? 

4.  Based  on  a  Southwest  Asia  scenario,  what  is  the  relative  importance  of  each 
factor  with  respect  to  the  others? 

These  questions  were  answered  through  a  three-phase  methodology  shown  in  Table  2  of 
Chapter  3. 

Phase  I  of  the  research  was  accomplished  through  a  literature  review  of  Joint  and 
Air  Force  operational  planning  publications,  and  through  Cognitive  Task  Analysis  (CTA) 
interviews  conducted  with  campaign  planning  staff  members  from  Headquarters  United 
States  Air  Force  (USAF),  Headquarters  USAF  Central  Command  (CENTAF),  Air 
Combat  Command,  and  Air  University.  The  purpose  of  these  interviews  was  to  elicit 
their  opinions  and  experience  on  contextual  issues  that  are  addressed  when  conducting 
crisis  action  planning.  The  results  of  the  literature  review  and  the  CTA  interviews  led  to 
the  development  of  a  hierarchy  of  campaign  specific  issues,  satisfying  the  first  research 
question. 

Phase  II  of  the  research  methodology  consisted  of  a  Value-Focused  Thinking 
(VFT)  interview  with  campaign  planners  from  CENTAF.  The  purpose  of  the  VFT 
interview  was  to  obtain  quantifiable  ranges  of  measurements  to  measurable  campaign 
specific  issues  within  the  context  of  a  SWA  Theater  crisis.  The  result  of  the  VFT 
interview  led  to  the  development  of  the  Excel  spreadsheet  based  decision  support  tool, 
satisfying  the  second  research  question. 

Phase  III  of  the  research  methodology  consisted  of  an  e-mail  and  interview  Delphi 
study  to  obtain  relative  weights  and  relationships  between  the  campaign  specific  issues 
identified  in  the  value  hierarchy.  Campaign  planners  previously  contacted  for  Phase  I 
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and  Phase  II  of  the  researeh  were  ineluded  in  the  Delphi  study.  The  result  of  the  Delphi 
study  led  to  the  development  and  confirmation  of  the  relative  weight  of  each  campaign 
specific  issue  as  they  relate  to  the  entire  value  hierarchy.  These  weights  were 
incorporated  into  the  decision  support  tool,  resulting  in  an  operational,  value-focused 
asset  set  assessment  tool.  The  results  of  the  Delphi  study  answered  the  third  and  fourth 
research  questions. 

Finally,  the  campaign  specific  issue  value  model  was  analyzed  based  upon 
notional  data  developed  for  use  within  the  Air  Force  Institute  of  Technology  (AFIT)/ALP 
asset  set  optimization  tool.  This  SWA  Theater  decision  support  tool  can  be  incorporated 
into  the  AFIT/ALP  optimization  tool,  providing  a  technique  to  assess  the  contextual 
issues  that  may  affect  the  selection  of  a  deployable  set  of  combat  aircraft. 

CONCLUSIONS 

COMBINATION  OF  COGNITIVE  TASK  ANALYSIS  AND  VALUE  FOCUSED 
THINKING  METHODOLOGIES 

This  research  has  shown  that  CTA  and  VFT  methodologies  can  be  synergistically 
combined,  improving  the  viability  of  which  a  decision  support  tool  can  be  developed  and 
accepted.  The  CDM  interviews  were  key  to  identifying  the  key  campaign-specifie  issues 
whieh  campaign  planners  must  make  decisions  about  when  selecting  combat  aircraft  to 
deploy  in  response  to  a  theater  crisis.  These  issues  may  be  easily  identified  and 
addressed  early  on  in  the  development  of  a  campaign  plan,  by  campaign  planners  with 
several  years  of  experience.  The  VFT  interviews  integrated  this  information,  assisting  in 
the  definition  of  measurement  functions  and  identification  of  weighting  between  the 
campaign-specific  issues.  The  combination  of  these  two  methodologies  will  assist  less 
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experienced  campaign  planners  in  identifying  and  addressing  these  operationally 
endangering  issues  earlier  on  in  the  development  of  a  campaign  plan,  and  will  help  more 
experienced  campaign  planners  to  do  a  better  job  quicker  and  more  efficiently  the  first 
time  around. 

DEVELOPMENT  OF  A  CAMPAIGN-SPECIFIC  ISSUE  DECISION  SUPPORT  TOOL 

It  was  possible  to  identify  a  set  of  campaign  specific  issues  from  Joint  and  Air 
Force  publications  and  through  CTA  interviews  with  current  Air  Force  campaign 
planners.  From  this  set  of  issues,  a  value  model  was  developed  to  study  the  affects  of 
contextual  issues  on  the  selection  of  a  set  of  combat  aircraft  to  respond  to  a  theater  crisis. 
The  model  was  designed  to  incorporate  currently  notional  data  from  the  AFIT/ALP  asset 
set  optimization  tool,  which  will  be  used  within  the  ALP  computational  architecture  to 
improve  the  development  of  deployment  plans. 

LIMITATIONS 

The  focus  of  this  research  has  been  to  examine  the  issues  or  factors  not  tied  to  the 
absolute  capability  of  United  States  Air  Force  (USAF)  combat  aircraft  to  perform 
specific  aerospace  missions,  that  would  affect  the  deployment  suitability  of  particular 
aircraft  assets  to  counter  a  theater  threat.  Because  of  this,  the  absolute  capability  of  each 
individual  USAF  combat  aircraft  was  not  considered.  Furthermore,  campaign  specific 
issues,  not  tied  to  the  absolute  capabilities  of  combat  aircraft  assets,  may  vary  from 
theater  to  theater;  therefore,  this  decision  support  tool  research  focuses  only  on  the 
Southwest  Asia  (SWA)  Theater.  The  Expeditionary  Aerospace  Force  concept  has  been 
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developed  to  serve  the  SWA  Theater.  Therefore,  the  development  of  a  decision  support 
to  based  upon  the  SWA  theater  can  be  most  effectively  applied  by  campaign  planners 
within  today’s  Concepts  of  Operations.  However,  the  campaign  specific  issues  identified 
in  this  research  may  be  important  in  other  theaters. 

A  further  limitation  of  this  research  is  the  limited  number  of  campaign  planning 
experts  contacted  to  participate  in  the  CTA  and  VFT  interviews  and  the  Delphi  study. 
Additional  experts,  amiable  to  the  development  of  a  campaign  specific  issue  decision 
support  tool  would  improve  the  understanding  and  support  of  this  research. 

RECOMMENDATIONS 

It  is  recommended  that  this  model  be  incorporated  into  the  AFIT/ALP  decision 
support  tool  for  selecting  deployable  sets  of  combat  aircraft.  The  addition  of  this  model 
provides  an  avenue  for  the  analysis  of  the  effect  of  campaign  specific  issues  on  the  value 
of  combat  aircraft. 

A  second  recommendation  from  this  research  is  with  respect  to  the  application  of 
the  Delphi  study  technique  to  elicit  a  consensus  from  a  group  of  individuals.  The 
viability  of  the  Delphi  study  may  have  limited  success  with  e-mail  based  surveys.  It  was 
seen  in  this  research  that  questionnaires  requiring  a  lengthy  time  commitment  or 
extensive  thought  to  complete  may  limit  response  rates,  if  those  questionnaires  are  based 
on  e-mail  attachments.  Questionnaires  limited  to  the  body  of  e-mail  received  a  higher 
response  rate  due  to  their  ease  of  use  and  readability.  Future  research  considering  the 
application  of  the  Delphi  methodology  in  an  e-mail  format  should  be  wary  of  this  limiting 
characteristic. 
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FUTURE  RESEARCH 


This  effort  should  be  expanded  to  other  operational  theaters  and  should  be 
developed  for  overall  USAF  use.  Expansion  of  the  model  will  include  the  confirmation 
of  the  campaign  specific  issues  identified  in  the  value  hierarchy  with  respect  to  all 
operational  theaters.  Relative  weights  of  the  issues  within  the  value  hierarchy  must  also 
be  developed  and  tested  for  both  different  operational  theaters  and  different  deployment 
scenarios.  It  is  believed  that  the  relative  weights  of  these  campaign  specific  issues  will 
undoubtedly  change  for  different  deployment  scenarios,  such  as  a  humanitarian, 
peacekeeping,  or  show  of  force  deployments. 

Another  potential  research  effort  includes  the  development  of  real-world  data  to 
study  comparable  asset  sets.  The  application  of  the  contextual  issue  decision  support  tool 
to  the  AFIT/ALP  asset  set  optimization  tool  is  dependent  upon  the  development  of  real- 
world  information  specific  to  combat  aircraft  and  the  campaign  specific  measures  and 
constraints. 

SUMMARY 

The  model  developed  for  the  Air  Force  Institute  of  Technology  (AFIT)/ Advanced 
Logistics  Project  (ALP)  decision  support  tool  provides  a  method  to  answer  the  question 
of  how  campaign  specific  issues  can  affect  the  selection  of  combat  aircraft  to  deploy  to 
the  Southwest  Asian  Theater,  consistent  with  Air  Force  values.  The  model  was 
developed  with  the  expert  help  of  campaign  planners  from  Headquarters  USAF, 
Headquarters  USAF  Central  Command,  and  Air  Combat  Command,  as  well  as  through 
current  Air  Force  and  Joint  planning  publications.  The  model  was  demonstrated  by 
scoring  50  unique  sets  of  assets  identified  within  the  AFIT/ALP  Pilot  Problem.  This 
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work  sets  the  foundation  for  further  work  and  partnership  between  AFIT,  ALP,  and  the 
Defense  Advanced  Research  Projects  Agency. 
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APPENDIX  A:  CTA  Interview  Bullet  Background  Paper 
Deployment  Planning  Factors 


1.  The  purpose  of  this  background  paper  is  to  provide  the  interviewee  with  an  overview 
of  discussion  topics  that  will  be  addressed  at  this  interview.  The  purpose  of  this 
interview  is  to  obtain  knowledge  and  understanding  of  the  factors  that  influence  the 
operational  effectiveness  of  aircraft  assets  in  deployed  location 

-  Air  Force  has  recently  moved  to  the  expeditionary  concept. 

--  Plan  to  deploy  a  small  contingent  of  aircraft,  36  aircraft,  2  to  6  bomber  assets 

—  Forces  may  be  expected  to  commence  required  aerospace  missions  immediately 
upon  arrival. 

--  Engaging  enemy  may  require  conducting  several  types  of  aerospace  missions, 
defined  in  Air  Force  Doctrine  1-1. 

-  Important  to  look  at  inherent  capabilities  of  aircraft  assets  when  selecting  specific  types 
for  deployment 

-  We  look  at  more  than  just  capabilities  of  individual  aircraft  assets  when  planning 

—  Aspects  of  a  deployment  scenario  may  make  selection  of  one  aircraft  type  more 
preferred  over  another  type,  or  may  negate  possibility  of  selecting  certain  types. 

—  Identification  and  understanding  of  these  aspects,  or  factors,  is  focus  of  today. 

-  Walk  me  through  planning  process  for  a  expeditionary  force  contingency  deployment 

—  Understand  several  layers  of  planning  and  decision-making  authority,  but  interested 
in  your  views  at  your  level 

—  In  determining  the  proper  aircraft  assets  to  be  deployed  and  planning  for  their 
deployment,  what  factors  do  you  look  at? 

-  Focus  on  identified  factors 

—  What  makes  these  important? 

—  Is  there  an5h;hing  that  may  affect  this  factor. .  .underlying  sub-factors? 

-  Focus  on  sub-factors  identified 
“  What  makes  these  important? 

—  Is  there  anything  that  may  affect  this  factor. .  .underlying  sub-factors? 

-  Continue  break  down  of  sub-factors  until  time  no  longer  permits,  or  until  interviewee 
has  exhausted  knowledge  base. 

-  Conclusion  to  interview 

—  Any  other  factors  we  have  not  identified  that  you  feel  are  important,  no  matter  level? 

—  Any  other  topics  you  feel  are  important  we  did  not  discuss? 

—  Any  contacts  you  know  of  that  I  should  talk  to  regarding  this  research? 
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-  In  a  few  weeks,  I  will  begin  second  phase  of  research,  and  I  would  like  to  request  your 
continued  support 

—  Second  phase  will  consist  of  several  e-mail  questionnaires 

—  Determine  ranges  of  values  and  relative  weights  of  importance  for  each  factor 
identified  and  discussed  today. 

-  Point  of  Contact  for  further  discussion 

—  ILt  Christopher  Buzo,  Air  Force  Institute  of  Technology  (AFIT) 
rChristopher.Buzo@afit.af  min 

~  Major  Alan  Johnson,  AFIT  (Alan ,  Johnson@afit.  af  mi  IT  DSN  255-3636  ext  4284 

—  Major  Steve  Swartz,  AFIT  (Stephen. Swartz@afit.af  mill 

“  Advanced  Logistics  Project  Website  (www. darpa. mi  1/iso/ alp) 
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APPENDIX  B:  CTA  Interview  Narrative 


INTRODUCTION: 

Good  Afternoon  and  thank  you  for  taking  time  out  of  your  busy  schedule  to  meet 
with  me.  As  you  may  know,  I  am  currently  conducting  research  into  the  factors  that 
influence  the  operational  effectiveness  of  aircraft  assets  in  a  deployed  location.  The 
purpose  of  this  interview  with  you  is  to  elicit  your  knowledge  and  understanding  of  these 
factors. 

The  information  I  collect  through  this  interview  will  be  combined  with  that  of 
others  and  reported  in  aggregate.  Therefore,  anything  that  is  obtained  through  this 
interview  is  confidential.  Other  than  my  observers  and  myself,  no  other  person  will  have 
access  to  any  identifiable  information. 

Understanding  this  obligation  of  confidentiality,  I  would  like  to  ask  your 
permission  to  record  this  conversation.  If  at  anytime  you  would  like  to  stop  recording  for 
any  reason,  please  just  let  me  know.  If  you  are  interested,  I  would  be  glad  to  forward  a 
copy  of  this  interview  to  you  after  it  is  transcribed. 

I  have  to  apologize  in  advance  for  having  to  watch  the  clock  and  perhaps  trying  to 
push  you  ahead  but  I  do  not  want  to  take  up  more  than  45  minutes  of  your  time  and  there 
are  a  number  of  things  I  would  like  to  cover. 

After  saying  that,  I  still  want  to  stress  that  the  interview  is  largely  unstructured. 
However,  if  there  is  anything  that  you  would  like  to  discuss  further  with  in  the  context  of 
these  issues,  just  let  me  know.  Now,  do  you  have  any  questions  before  we  start? 

BACKGROUND: 

As  we  begin  our  discussion  of  influential  factors,  I  would  like  to  set  the 
deployment  stage.  As  you  know,  the  Air  Force  has  recently  moved  to  the  expeditionary 
concept.  Under  an  Air  Expeditionary  deployment,  we  plan  to  deploy  a  small  contingent 
of  aircraft,  generally  36  fighter  aircraft,  2  to  6  bomber  assets,  and  other  necessary  support 
aircraft.  As  you  know,  these  forces  may  be  expected  to  commence  required  aerospace 
missions  immediately  upon  arrival.  Engaging  the  enemy  may  require  conducting  several 
types  of  aerospace  missions,  as  defined  in  Air  Force  Doctrine  1-1. 

Research  is  currently  being  conducted  in  an  effort  to  assist  theater  commanders 
and  campaign  planners  in  selecting  sets  of  aircraft  assets  (mixes  of  fighters  and  bombers) 
that  will  provide  the  best  “utility”  to  the  commander,  based  upon  the  theater 
commander’s  aerospace  mission  requirements.  Using  the  developed  mathematical 
program,  asset  sets  will  be  identified  to  provide  the  commander  with  the  best  “match-up” 
of  aircraft  assets  to  required  missions.  These  match-ups  are  based  upon  the  inherent 
capabilities  of  each  individual  aircraft  asset  to  specific  aerospace  missions.  Of  course, 
we  know  that  we  look  at  more  than  just  the  capabilities  of  individual  aircraft  assets  when 
planning  a  deployment.  There  are  aspects  of  a  deployment  scenario  that  may  make  the 
selection  of  one  aircraft  type  more  preferred  over  another  type,  or  may  negate  the 
possibility  of  selecting  one  or  more  types.  The  identification  and  understanding  of  these 
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aspects,  or  factors,  is  what  I  would  like  to  focus  on  today.  I  am  going  to  ask  you  a  series 
of  questions  aimed  at  obtaining  your  views  and  insights  into  these  deployment  factors. 

ILICITATION  QUESTIONS: 

1.  I  would  like  you  to  walk  me  through  the  planning  process  for  a  contingency 
deployment;  one  that  would  be  suited  to  an  expeditionary  sized  deployment.  I  understand 
that  there  are  several  layers  of  planning  and  decision-making  authority  within  the 
planning  process,  but  right  now  I  am  interested  in  your  views  at  your  level  of  the 
planning  process.  In  determining  the  proper  aircraft  assets  to  be  deployed,  what  factors 
do  you  look  at? 

-  If  answer(s)  is/are  not  specific,  reiterate  question 

-  Continue  to  press  for  answers  until  interviewee  appears  to  be  out  of  responses 

2.  Are  there  any  more  details  or  factors,  no  matter  how  insignificant  you  may  feel  they 
are,  that  should  be  identified? 

3.  INSERT  APPRECIATIVE  REMARK.  Now  I  would  like  to  focus  in  on  these  factors 
that  you  have  identified.  If  at  any  time  you  think  of  another  factor  that  has  not  been 
previously  identified,  no  matter  how  insignificant  you  feel  it  may  be,  please  bring  it  up. 

Looking  at  (EACH  INDIVIDUAL  IDENTIFIED  FACTOR),  what  makes  this 
factor  important  in  the  planning  process?  Is  there  anything  that  may  affect  this 
factor?  Are  there  any  underlying  sub-factors? 

4.  It  looks  as  though  we  have  identified  some  very  important  factors  in  planning  an 
expeditionary  force  deployment,  and  we  have  found  several  underlying  sub-factors  that 
can  influence  the  decision.  I  would  now  like  to  focus  on  these  sub-factors.  Once  again, 
if  at  any  time  you  think  of  another  factor  that  has  not  been  previously  identified,  no 
matter  how  insignificant  you  feel  it  may  be,  please  bring  it  up. 

Looking  at  (EACH  INDIVIDUAL  IDENTIFIED  SUB-FACTOR),  what  makes 
this  sub-factor  important  in  the  planning  process?  Is  there  anything  that  may 
affect  this  sub-factor?  Are  there  any  underlying  sub-factors? 

5.  I  would  like  to  continue  this  break  down  of  sub-factors.  Looking  at . 

CONCLUSION; 

6.  It  appears  as  though  we  have  identified  a  number  of  important  factors  that  must  be 
considered  when  planning  an  expeditionary  force  deployment.  Do  you  feel  confident  that 
we  have  exhausted  this  list?  If  not,  are  there  any  factors  we  have  not  identified  that  you 
feel  are  important?  Was  there  anything  that  we  did  not  talk  about  that  seems  relevant  to 
this  research?  Are  there  any  people  or  offices  you  know  of  that  I  should  talk  to  regarding 
this  research? 
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7.  I  would  like  to  thank  you  very  much  for  your  time  for  this  interview.  In  a  few  weeks, 
I  will  begin  the  second  phase  of  this  research,  and  I  would  like  to  request  your  continued 
support  for  this  phase.  The  second  phase  of  my  research  will  consist  of  several  e-mail 
questionnaires  in  which  I  will  ask  several  planning  experts  (such  as  yourself)  to 
determine  ranges  of  values  and  relative  weights  of  importance  for  each  of  these 
deployment  planning  factors.  Again,  it  has  been  a  pleasure  to  talk  with  you  and  I  look 
forward  to  your  continued  support  in  this  research  effort. 
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APPENDIX  C:  VFT  Study  Dictionary  of  Campaign  Specific  Issues 

DEPLOYMENT  PLANNING  FACTOR  DEFINITIONS 


Title:  Multi-National  Compatibility 

Description:  Extent  to  which  host-nation  assets,  to  include  combat  aircraft,  equipment, 
spares,  and  support  equipment  can  be  incorporated  along  side  and  into  USAF  combat 
aircraft. 

Measurement:  “Compatibility  of  Assets  in  Asset  Set.”  0  -  100  percent.  At  100  percent 
compatibility,  the  staging  base  that  the  host  nation  maintains  and  operates  like  assets  for 
100  percent  of  the  assets  in  the  asset  set.  USAF  combat  aircraft  can  be  incorporated  into 
the  staging  base  without  bringing  in  testing  equipment,  spares,  or  excessive  maintenance 
equipment.  At  0  percent,  all  of  the  assets  in  the  asset  set  are  different  (no  like  assets 
between  United  States  Air  Force  combat  aircraft  and  host  nation  combat  aircraft).  In  this 
situation,  campaign  planners  would  need  to  bring  a  complete  supply  of  testing  equipment, 
spares,  and  maintenance  and  repair  equipment  for  all  combat  aircraft  brought  into  the 
staging  base.  The  entire  range  of  percentages  of  host-nation  compatibility  is  possible. 


Title:  Beddown  Location 

Description:  The  effectiveness  of  an  individual  asset  within  the  asset  set  is  dependent 
upon  the  distance  from  the  staging  base  to  the  target. 

Measurement:  “Distance  (miles)  From  Target.”  0  -  6000  miles.  This  measure  scores 
the  location  of  the  staging  base  from  the  general  enemy  target  area.  Bomber  and  fighter 
assets  have  different  distance  abilities;  therefore,  each  has  a  separate  curve.  Additionally, 
separate  curves  are  required  for  refueled  and  un-refueled  operations.  It  has  been 
determined  for  this  analysis  that  6000  miles  is  the  effective  distance  of  bomber  assets  due 
to  crew  capabilities.  The  effective  range  of  a  fighter  aircraft,  determined  for  this  analysis, 
is  2500  miles,  due  to  crew  capabilities.  For  each  type  of  aircraft  (fighter  or  bomber),  the 
entire  range  of  effective  distances  within  their  specific  ranges  is  possible. 
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Title:  Re-supply  Ability 

Description:  Continued  combat  operations  from  the  staging  base  are  dependent  upon  the 
ability  of  USAF  force’s  equipment  and  materials  to  be  resupplied.  Assuming  the  staging 
base  has  been  predetermined,  the  selection  of  combat  aircraft  assets  will  be  dependent 
upon  individual  asset’s  resupply  requirements.  Combat  aircraft  that  require  smaller 
amounts  of  POL,  munitions,  and  other  equipment  will  have  a  higher  inherent  value  than 
an  aircraft  type  that  requires  more  assets. 

Measurement:  “Ratio  of  Resupply  to  Consumption  to.”  0-1.  At  the  ratio  of  resupply 
to  consumption  increases  to  1,  US  force’s  ability  to  resupply  the  staging  base  with 
required  equipment  and  materials  (i.e.  munitions,  aircraft  replacement  parts)  for  an 
aircraft  asset  is  equal  to  the  consumption  of  those  same  equipment  and  materials.  As  the 
ratio  of  resupply  to  consumption  decreases  to  0,  combat  aircraft  consumption  rates  far 
exceed  our  force’s  ability  to  resupply  the  staging  base.  Ratios  above  1  would  not  be 
experienced,  as  they  imply  assets  are  being  shipped  to  the  staging  base  location  that  are 
not  needed.  The  entire  range  of  ratios  of  resupply  to  consumption  is  possible. 


Title:  Munitions  Storage:  Fighter  Assets 

Description:  Continued  fighter  operations  depend  heavily  on  the  pre-determined  staging 
base’s  ability  to  store  specific  munitions  for  fighter  operations.  Fighter  aircraft  can  only 
bring  one  load  of  munitions;  additional  sorties  require  the  ability  of  the  staging  base  to  re¬ 
load  the  fighter. 

Measurement:  “Fighter  Sorties  per  Day.”  0  -  100  percent.  At  100  percent  sortie  rate 
per  day,  the  staging  base’s  munitions  storage  area  (MSA),  (includes  storage  and  supply) 
can  support  100  percent  of  the  Commander  in  Chief  s  (CINCs)  fighter  sortie 
requirements  per  day.  At  0  percent  sortie  rate  per  day,  no  sorties  can  be  supported. 

While  0  percent  is  theoretically  possible,  0  percent  is  not  likely,  because  if  the  staging 
base  could  not  store  the  required  munitions  for  a  particular  fighter  sortie,  another  aircraft 
alternative  would  be  selected.  At  50  percent  sortie  rate  per  day,  only  50  percent  of  the 
sorties  required  of  the  particular  aircraft  asset  can  be  conducted.  The  entire  range  of 
percentages  of  fighter  sorties  per  day  supported  is  possible. 


106 


Title:  Munitions  Storage;  Bomber  Assets 


Description:  Continued  bomber  operations  depend  on  the  pre-determined  staging  base’s 
ability  to  store  specific  munitions  for  bomber  operations.  Bomber  aircraft  can  only  bring 
one  load  of  munitions;  additional  sorties  require  the  ability  of  the  staging  base  to  re-load 
the  fighter. 

Measurement;  “Bomber  Sorties  per  Day.”  0-100  percent.  At  100  percent  sortie  rate 
per  day,  the  staging  base’s  munitions  storage  area  (MSA),  (includes  storage  and  supply) 
can  support  100  percent  of  the  Commander  in  Chief  s  (CINCs)  bomber  sortie 
requirements  per  day.  At  0  percent  sortie  rate  per  day,  no  sorties  can  be  supported. 

While  0  percent  is  theoretically  possible,  0  percent  is  not  likely,  because  if  the  staging 
base  could  not  store  the  required  munitions  for  a  particular  bomber  sortie,  the  bomber 
aircraft  would  re-locate  to  another  staging  base  or  another  aircraft  alternative  would  be 
used.  At  50  percent  sortie  rate  per  day,  only  50  percent  of  the  sorties  required  of  the 
particular  aircraft  asset  can  be  conducted.  The  entire  range  of  percentages  of  bomber 
sorties  per  day  supported  is  possible. 


Title:  Petroleum,  Oil,  and  Lubricants  (POL)  Capability 

Description:  The  staging  base’s  POL  system  includes  storage  and  supply  from  both 
military  and  host-nation  contracted  sources.  POL  capability  for  a  particular  asset  set 
must  take  into  consideration  the  per-sortie  POL  requirements  of  each  individual  aircraft 
in  the  asset  set  as  well  as  the  daily  sortie  requirements  for  each  individual  aircraft. 

Measurement:  “Mission  Capability.”  0  -  100  percent.  At  100  percent  mission 
capability,  the  staging  base  POL  system  can  support  100  percent  of  the  Commander  in 
Chiefs  (CINC’s)  sortie  requirements,  whether  the  POL  comes  from  on-base  or  off-base 
contracted  sources.  At  0  percent  mission  capability,  no  sorties  can  be  supported.  While  0 
percent  is  theoretically  possible,  if  0  percent  capability  were  experienced,  aircraft  assets 
would  be  directed  to  a  different  staging  base.  The  entire  range  of  percentages  of  mission 
capability  due  to  POL  storage  and  supply  is  possible. 
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Title;  Tanker  Support  Required 

Description;  Depending  upon  the  location  of  the  staging  base  from  the  enemy  target  and 
the  fuel  requirements  per  aircraft  asset  sortie,  and  the  number  of  aircraft  assets 
conducting  sorties  at  a  particular  time,  multiple  tanker  assets  may  be  required. 

Measurement;  “Tankers  Required.”  0-10.  This  measure  scores  the  number  of  tankers 
required  in  the  theater  to  support  asset  set  sorties.  Depending  upon  the  fuel  requirements 
for  a  particular  aircraft  asset  and  the  distance  between  the  staging  base  and  enemy  target, 
tanker  assets  may  be  required.  Additionally,  the  intensity  of  combat  in  and  around  the 
enemy  target  may  require  tanker  support.  Note,  this  measure  only  counts  tanker  that  are 
specifically  dedicated  to  combat  operations;  cargo  and  intelligence  gathering  aircraft  are 
not  considered.  This  measure  counts  actual  numbers  of  tankers  required  over  the  area  of 
responsibility,  and  not  tanker  sorties.  No  tankers  required  (0)  would  be  experienced  if 
combat  operations  did  not  require  aerial  refueling.  Ten  tankers  required  (10)  would  be 
experienced  if  combat  operations  required  an  extensive  amount  of  sustained  aerial 
operations.  The  entire  range  of  number  of  tankers  required  is  possible,  although  10 
tankers  required  are  a  theoretical  maximum. 


Title;  Topography 

Description;  The  extent  to  which  natural  and  man-made  land  formations  around  the  pre¬ 
determined  staging  base  effects  the  ability  of  individual  aircraft  assets  within  an  asset  set 
to  take-off  and  land  at  the  staging  base. 

Measurement;  “Degradation  of  Mission.”  0  -  100  percent.  At  100  percent  mission 
degradation,  no  sorties  can  fly  in  or  out  of  the  staging  base  due  to  topographical 
restrictions.  100  percent  mission  degradation,  is  a  theoretical  maximum,  because  if  all 
aircraft  assets  could  not  take-off  from  the  staging  base  with  enough  munitions  to 
successfully  accomplish  the  mission,  those  aircraft  would  be  directed  to  a  different 
staging  base.  At  0  percent  mission  degradation,  all  aircraft  can  fly  sorties  out  of  and 
return  to  the  staging  base  with  enough  munitions  and  fuel  to  successfully  complete  the 
required  aerospace  missions.  At  50  percent  mission  degradation,  combat  aircraft  can  fly 
into  and  out  of  the  staging  base  at  50  percent  capability.  50  percent  capability  is  defined 
as  an  aircraft  asset  that  can  only  be  configured  with  enough  fuel  and  munitions  to  achieve 
a  50  percent  probability  of  kill  over  the  enemy  target.  The  entire  range  of  percentages  of 
degradation  of  mission  due  to  staging  base  location  topography  is  possible. 
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Title:  Climate 


Description:  The  extent  to  which  heat,  humidity,  adverse  wind  and  atmospheric  pressure 
would  affect  the  ability  of  particular  aircraft  assets  to  conduct  combat  operations. 

Measurement:  “Degradation  of  Mission.”  0  -  100  percent.  At  100  percent  Mission 
Degradation,  no  sorties  can  fly.  This  is  a  theoretical  maximum  degradation  of  mission, 
because  if,  for  instance,  the  cross  winds  at  the  staging  base  were  such  that  a  particular 
aircraft  asset  could  not  land,  that  aircraft  asset  would  be  directed  to  another  base.  At  0 
percent  Mission  Degradation,  all  sorties  (all  aircraft)  can  fly  at  100  percent  capability.  At 
50  percent  Mission  Degradation,  An  aircraft  asset  can  only  take-off  at  50  percent  cll 
sorties  (all  aircraft)  can  fly  at  50  percent  capability  (i.e.  only  50  percent  of  maximum 
munitions  loading  for  bombers).  The  entire  range  of  percentages  of  mission  degradation 
due  to  climate  is  possible 


Title:  Availability  of  Assets 

Description:  Extent  to  which  aircraft  assets  in  the  asset  set  are  operationally  capable  of 
being  used  to  conduct  aerospace  missions  required  by  the  theater  commander. 

Measurement:  “Average  Mission  Capable  Rate  of  Assets  in  Set.”  0-100  percent.  At 
100  percent  average  mission  capable  rate,  all  of  the  assets  in  the  set  of  assets  have  a  100 
percent  mission  capable  rate.  This  would  mean  that  maintenance  and  repair  requirements 
for  each  individual  aircraft  in  the  asset  set  could  be  completed  without  reducing  the 
Mission  Capability  rate.  At  0  percent  average  mission  capable  rate,  all  assets  in  the  set  of 
assets  are  non-flyable.  This  should  be  a  theoretical  minimum,  because  under 
circumstances  where  all  of  the  assets  in  the  asset  set  were  grounded  due  to  maintenance, 
another  set  of  aircraft  would  be  chosen.  At  50  percent  average  mission  capable  rate, 
some  assets  may  have  a  mission  capable  rate  higher  than  50  percent,  while  some  assets 
may  have  a  mission  capable  rate  less  than  50  percent.  The  entire  range  of  percentages  of 
availability  of  assets  is  possible. 
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Title:  Asset  Mission  Flexibility 

Description:  Extent  to  which  the  aircraft  within  the  asset  set  can  perform  all  of  the 
possible  aerospace  missions  defined  in  AFM  1-1  and  AFDD  2-1.  The  relative  worth  of 
the  asset  set  would  be  high  if  that  asset  set  could  perform  any  mission  required  by  the 
theater  commander. 

Measurement:  “Percent  Capability  of  All  Aerospace  Missions.”  0  -  100  percent.  AtO 
percent  aerospace  mission  capability,  the  assets  in  the  asset  set  cannot  perform  any 
aerospace  combat  mission  required  by  the  theater  Commander  in  Chief  (CINC).  This  is  a 
theoretical  minimum,  as  all  aircraft  assets  are  capable  of  performing  at  least  one 
aerospace  mission.  At  100  percent  aerospace  mission  capability,  the  assets  in  the  asset 
set  can  perform  any  and  all  of  the  aerospace  combat  missions  that  could  possibly  be 
required  by  the  theater  CINC.  100  percent  aerospace  mission  capability  is  achieved  if  the 
assets  in  the  asset  set  can  collectively  perform  all  of  the  aerospace  combat  missions  as 
defined  in  Air  Force  Manual  1-1  and  Air  Force  Doctrine  Document  2-1.  The  entire  range 
of  percentages  of  aerospace  mission  capability  is  possible. 


Title:  Asset  Set  Utilization 

Description:  Extent  to  which  aircraft  in  the  asset  set  is  used  to  perform  aerospace 
missions  as  required  by  the  theater  commander.  This  assumes  that  aircraft  brought  into 
the  theater  will  be  used  to  conduct  combat  sorties  and  are  not  brought  in  only  as  a  show 
of  force.  If  aircraft  assets  are  brought  in  only  as  a  show  of  force,  this  may  or  may  not  be 
an  issue. 

Measurement:  “Use  of  Primary  Aircraft  Authorized.”  0-70  percent.  At  0  percent  use 
of  primary  aircraft  authorized,  no  assets  in  the  asset  set  are  being  used.  This  is  a 
theoretical  minimum,  because  assuming  the  aircraft  are  not  brought  in  only  as  a  show  of 
force,  the  aircraft  should  be  used  for  at  least  pilot  training.  At  70  percent  use  of  primary 
aircraft  authorized,  all  of  the  assets  in  the  asset  set  are  being  used  to  their  peak 
operationally  safe  use.  100  percent  use  of  primary  aircraft  authorized  is  defined  as  the 
continual  “turn  and  burn”  of  all  aircraft  assets  at  the  staging  base.  Aircraft  would  be  at 
the  staging  base  only  long  enough  to  reload  munitions  and  change  pilots,  if  necessary.  70 
percent  has  been  determined  to  be  the  maximum  rate  an  aircraft  asset  can  be  safely  used 
before  the  long-term  operational  capability  is  jeopardized.  The  entire  range  of 
percentages  of  use  of  primary  aircraft  authorized  is  possible. 
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Title:  Asset  Set  Over-Utilization 


Description;  Extent  to  which  aircraft  in  the  asset  set  is  over-used  to  perform  aerospace 
missions  as  required  by  the  theater  commander.  This  assumes  that  aircraft  brought  into 
the  theater  are  being  used  beyond  their  operationally  safe  limit.  This  measure  is  an 
extension  of  the  Asset  Set  Utilization  measure  whereby  aircraft  assets  are  being  used 
beyond  the  measure’s  range. 

Measurement:  “Use  of  Primary  Aircraft  Authorized.”  70  -  100  percent.  At  70  percent 
use  of  primary  aircraft  authorized,  all  assets  in  asset  set  are  being  used  to  their  peak 
operational  efficiency.  This  could  be  experienced  during  an  initial  surge  of  sortie 
activity,  where  USAF  combat  aircraft  have  been  tasked  to  halt  enemy  action.  At  100 
percent  used  of  primary  aircraft  authorized,  aircraft  in  asset  set  have  been  surged  to 
theoretical  maximum,  whereby  all  of  the  aircraft  in  the  asset  set  are  in  a  state  of  continual 
“turn  and  burn.”  100  percent  use  of  primary  aircraft  authorized  cannot  be  sustained,  as  it 
will  quickly  degrade  the  capability  of  the  aircraft  assets  to  perform  the  mission.  The 
entire  range  of  percentages  of  use  of  primary  aircraft  authorized  is  possible. 


Title:  Enemy  Ability  and  Capabilities  versus  Asset  Set 

Description:  This  factor  measures  the  comparison  of  enemy  assets  to  USAF  combat 
aircraft  assets.  Assuming  US  intelligence  can  determine  the  extent  of  the  enemy’s 
capability,  this  measure  shows  the  relative  worth  of  the  amount  of  aircraft  assets  brought 
into  the  pre-determined  staging  base.  One  issue  of  this  measure  is  the  number  of  USAF 
combat  aircraft  brought  into  the  staging  base;  another  issue  is  the  capabilities  of  the 
enemy  to  counter  the  USAF  combat  aircraft  brought  into  the  staging  base. 

Measurement:  “Ratio  of  Allied  to  Enemy  Forces.”  0-3.  As  the  ratio  of  Allied  to 
Enemy  Forces  approaches  0,  the  enemy’s  ability  and  capability  versus  the  combined 
capabilities  of  the  combat  aircraft  assets  in  the  asset  set  is  continuing  to  increase.  This 
means  that  the  capabilities  of  the  enemy  exceed  USAF  combat  aircraft  capabilities.  For 
example,  the  capability  of  a  single  enemy  MIG  may  have  the  same  capabilities  as  3  US 
F-15s.  As  the  ratio  of  Allied  to  Enemy  Forces  approaches  1,  the  capability  of  enemy 
forces  and  USAF  combat  aircraft  is  equal.  For  example,  the  capability  of  a  single  enemy 
MIG  may  have  the  same  capabilities  as  one  US  F-16.  As  the  ratio  of  Allied  to  Enemy 
Forces  approaches  3,  the  capabilities  of  USAF  combat  aircraft  outweighs  the  capability 
of  enemy  forces  by  300  percent.  For  example,  the  capability  of  three  enemy  MIGs  may 
have  the  same  capabilities  as  a  single  F-1 17.  The  entire  range  of  ratios  is  possible,  and 
while  the  ratio  of  USAF  combat  aircraft  to  Enemy  forces  may  exceed  3,  no  additional 
value  is  obtained.  Note:  this  is  a  capabilities  based  measure  that  requires  intelligence  on 
the  capabilities  and  number  of  the  enemy’s  forces  in  relation  to  the  USAF  combat  aircraft 
within  a  particular  asset  set. 
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Title:  Enemy  Intelligence,  Surveillance,  and  Reconnaissance  (ISR) 

Description;  This  measure  captures  the  enemy’s  ability  to  see  or  obtain  information  on 
US  activity  at  the  pre-determined  staging  base.  Enemy  ISR  could  be  obtained  through 
aerial  or  satellite  photography,  local  sympathizers  monitoring  US  activity,  or  spies  within 
the  host-nation  working  within  or  around  the  staging  base. 

Measurement:  “Activity  Exposure.”  0  -  100  percent.  At  0  percent  exposure,  the  enemy 
has  no  intelligence  on  or  ability  to  see  our  operations.  This  may  occur  due  to  the  enemy’s 
lack  of  ISR  capabilities  or  due  to  strict  secrecy  of  operational  activity  at  the  staging  base. 
At  100  percent  exposure,  the  enemy  has  full  knowledge  or  visibility  of  our  operations. 

100  percent  exposure  is  more  conceivable  than  0  percent  exposure,  as  the  presence  of  US 
combat  aircraft  at  a  staging  base  not  usually  hosting  US  forces  would  arouse  some 
suspicion  and  concern  of  enemy  leadership.  The  entire  range  of  percentages  of  activity 
exposure  is  possible,  due  to  the  capabilities  of  the  enemy  and  the  amount  of  local 
sympathy  for  the  enemy. 


Title:  Enemy  Interference 

Description:  This  measure  captures  the  enemy’s  ability  to  interfere  with  US  operations 
at  the  pre-determined  staging  base.  Enemy  interference  is  any  action  on  the  part  of  the 
enemy  that  would  hamper  US  combat  operations.  Enemy  interference  could  range  from 
picketing  and  loitering  around  the  staging  base,  isolated  attempts  to  gain  access  into  the 
base  and  base  operations,  bomb  threats,  attacks  on  aircraft  taking  off  and  landing  at  the 
base,  or  attacks  on  the  base  itself 

Measurement:  “Operation  degradation.”  0-100  percent.  At  0  percent  operation 
degradation,  no  enemy  interference  of  any  kind  is  experienced  at  the  staging  base.  At 
1 00  percent  operation  degradation,  enemy  interference  has  halted  any  and  all  combat 
operations,  to  include  destruction  of  the  staging  base.  100  percent  operation  degradation 
is  a  theoretical  maximum,  because  if  an  actual  attack  on  the  staging  base  were  expected 
to  occur,  the  aircraft  would  be  re-located  to  a  different  staging  base.  The  entire  range  of 
percentages  of  operation  degradation  is  possible. 


112 


Score 


APPENDIX  D:  Campaign  Specific  Issue  Graphs 


Multi-National  Support 


Percentage  Compatibility  of  Assets 

FIGURE  12;  Multi-National  Support  Value  Graph 


FIGURE  13:  Beddown  Location  -  Fighter  Distance  Value  Graph 


Bomber  Distance 


Refueled  Miles  From  Target 


FIGURE  14;  Beddown  Location  -  Bomber  Distance  Value  Graph 


Ability  to  Resupply 


0  500  1000  1500  2000  2500 

M  lies  From  POD 

FIGURE  15:  Resupply  Location  Value  Graph 
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FIGURE  16:  Munitions  Storage  Area  -  Fighter  Support  Value  Graph 


FIGURE  17;  Munitions  Storage  Area  -  Bomber  Support  Value  Graph 


POL  Capability 


FIGURE  18:  Petroleum,  Oils,  and  Lubricants  Capability  Value  Graph 


Tanker  Support  Required 


FIGURE  19:  Tanker  Support  Required  Value  Graph 
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Topography 


Percent  of  Mission  Degraded 

FIGURE  20:  Topography  Value  Graph 
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FIGURE  21:  Climate  Value  Graph 
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FIGURE  22:  Asset  Mission  Flexibility  Value  Graph 


Asset  Set  Utilization 


Peroent  Utility  of  All  Assets  in  Set 

FIGURE  23:  Asset  Set  Utilization  Value  Graph 


Score 


Average  MC  Rate  (%)  Of  Assets  in  Set 

FIGURE  24;  Availability  of  Assets  Value  Graph 


Eneny  Vs.  USAF  Capability 
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Rationed  to  Bienv  Rxces 

FIGURE  25:  Enemy  Vs.  USAF  Capability  Value  Graph 


Score  9  Score 


APPENDIX  E:  VFT  Methodology  Example 


Say,  for  example,  that  you  are  interested  in  purchasing  a  new  four-door,  family 
size  car.  There  are,  of  course,  many  styles  and  brands  to  choose  from,  each  having  many 
different  features.  You  do,  however,  have  three  different  cars  that  you  would  consider 
purchasing,  but  since  you  don  not  really  have  a  clear  favorite  between  these  three, 
choosing  the  best  vehicle  seems  a  bit  overwhelming.  The  first  thing  we  need  to  do, 
therefore,  is  sit  down  and  make  a  list  of  requirements.  You  know  you  can  afford  up  to 
$300  a  month  in  car  payments,  but  of  course,  the  less  you  have  to  spend  the  better  off  you 
will  be.  You  want  good  gas  mileage,  the  higher  the  better.  Air-Conditioning  and  Anti- 
Lock  breaks  would  be  nice  to  have.  You  have  now  created  a  list  of  factors,  which  you 
will  use  to  guide  your  car-buying  decision. 

Now  we  need  to  develop  quantitative  ranges  of  values  for  each  of  the  factors. 

You  already  know  that  you  can  only  pay  up  to  $300  a  month  for  your  new  car.  Any  car 
with  a  price  above  that  you  cannot  afford.  This,  then,  is  your  high  value  for  cost.  You 
then  decide  that  you  would  be  happiest  if  you  could  only  pay  $200  a  month  for  your  new 
car.  You  would  not  really  be  any  happier  paying  less  than  $200,  but  you  would  be  less 
happy  if  you  had  to  pay  more.  We  can  now  begin  to  graph  your  financial  requirements 
for  your  new  car. 


After  some  thought,  you  realize  that  if  the  cost  of  your  potential  new  car  was,  say, 
$240,  your  “level  of  happiness”  would  be  50-percent.  You  now  have  a  value  function  for 
the  cost  of  your  new  car,  with  values  ranging  from  0  (you  would  have  to  pay  $300/month 
for  your  new  car)  to  1  (you  could  pay  only  $200/month  for  your  new  car). 
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Now  you  need  to  do  the  same  value  analysis  for  each  of  your  remaining  factors: 
Gas-mileage,  air-conditioning,  and  anti-lock  breaks.  After  careful  thought,  you’ve 
developed  your  gas-mileage  function  as  follows: 


You  note  that  the  air-conditioning  and  anti-lock  break  factors  cannot  really  be 
graphed  the  same  way  that  cost  and  mileage  were.  Either  the  car  in  question  has  anti¬ 
lock  breaks,  or  it  does  not.  The  same  goes  for  the  A/C.  Therefore,  you  assign  a  value  of 
1  to  the  anti-lock  break  factor  if  the  car  does  have  anti-lock  brakes  and  a  value  of  0  if  the 
car  does  not  have  anti-lock  breaks.  The  same  value  function  is  developed  for  A/C.  If  the 
car  has  A/C,  the  factor  is  assigned  a  value  of  1;  if  not,  the  factor  is  assigned  a  value  of  0. 

Now  that  you  have  your  value  functions  for  each  of  your  factors,  you  need  to 
weight  the  individual  value  functions.  Are  you  equally  concerned  about  each  of  the 
factors  you  determined  were  important?  Probably  not.  If  you  are  like  me,  you  are 
probably  more  concerned  about  cost  than  any  of  the  other  factors.  For  this  example,  say 
that  you  are  three-times  as  concerned  about  the  cost  of  the  car  as  you  are  about  its  gas- 
mileage.  You  believe  that  having  anti-lock  breaks  is  twice  as  important  as  getting  good 
gas-mileage,  and  that  good  gas-mileage  and  air-conditioning  are  equally  important.  We 
can  now  put  this  into  quantitative  numbers,  because  your  weights  must  add  up  to  1  (100- 
percent).  Here’s  the  mathematics  to  this: 

Cost  =  3  X  Gas 

Gas  =  AC 

Breaks  =  2  x  Gas 

Cost  +  Gas  +  AC  +  Breaks  =  1  (this  must  be  true)  Therefore. . . 

(3  X  Gas)  +  Gas  +  Gas  +  (2  x  Gas)  =  1 

Gas  =  1/7 

Therefore,  Gas  =  1/7,  Cost  =  3/7,  AC  =  1/7,  and  Breaks  =  2/7.  You  have  now 
weighted  the  importance  of  each  one  of  your  factors  with  respect  to  each  other.  Now 
comes  the  time  for  some  research.  You  pick  up  the  latest  edition  of  the  Car  Trader  and 
read  up  on  thee  three  cars  you  have  your  eye  on.  You  write  down  the  figures  relevant  to 
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your  factors  for  each  car.  For  this  example,  let’s  assume  that  the  interest  rates  and 


Car  A 

Car  B 

CarC 

Cost/Month 

$200 

$250 

$300 

Gas-Mileage 

30MPG 

25  MPG 

30  MPG 

Anti-Lock  Breaks? 

No 

Yes 

Yes 

A/C? 

No 

No 

Yes 

Now  it  is  time  to  put  the  figures  from  your  research  into  your  value  functions. 
Compare  each  of  the  figures  for  each  car  to  their  respective  value  functions.  For 
example,  the  value  function  developed  for  cost  states  that  $200/month  has  a  value  of  1, 
and  $300/month  has  a  value  of  0.  We  can  match  the  car  figures  to  our  value-functions 
and  develop  a  new  chart. 


Car  A 

Car  B 

CarC 

Cost/Month 

1.0 

0.5 

0 

Gas-Mileage 

0.50 

0.25 

0.50 

Anti-Lock  Breaks? 

0 

1.0 

1.0 

A/C? 

0 

0 

1.0 

Now  lets  add  the  weighting  of  importance  to  each  of  the  factors  in  the  same  chart. 


Weights 

Car  A 

CarB 

Car  C 

Cost/Month 

3/7 

1.0 

0.5 

0 

Gas-Mileage 

1/7 

0.50 

0.25 

0.50 

Anti-Lock  Breaks? 

2/7 

0 

1.0 

1.0 

A/C? 

1/7 

0 

0 

1.0 

Now  that  we  have  all  the  numbers  we  need,  we  can  determine  the  value  of  Cars  A,  B,  and 
C  by  multiplying  each  factor  value  by  it’s  weight  and  sum  these  numbers.  Using  Car  A 
as  the  example, 

(3/7)*  1.0  +  (l/7)*0.50  +  (2/7)*0  +  (l/7)*0  =  0.50  -  Value  of  Car  A. 

We  can  sum  up  the  individual  factor  values  for  each  car  to  get  the  value  of  each  car; 

Car  A  =  0.50 
Car  B  =  0.53 
Car  C  =  0.50 

Therefore,  you  have  determined  Car  B  is  the  best  car  for  you,  because  it  has  the  highest 
value  compared  to  the  other  2  alternatives.  You  have  made  this  decision  based  upon: 

The  factors  you  determined  were  important  in  your  new  car 

The  range  of  figures  that  each  factor  could  have  and  their  corresponding  value 

The  weighting  of  importance  for  each  factor,  and 

The  3  cars  available  to  choose  from. 
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APPENDIX  F:  DELPHI  Study  Questionnaire  One 
INSTRUCTIONS 


Please  answer  all  questions  on  this  document  and  return  within  an  e-mail  to: 

Chrisotopher.Buzo@afit.  af.  mil 

1 .  Identification:  Personal  information  WILL  NOT  be  used  during  the  Delphi  process 
itself,  nor  will  your  comments  be  specifically  linked  to  you  in  my  thesis.  However,  your 
identification  is  relevant  to  the  research  effort  and  will  be  included  in  the  thesis. 
Name/Rank: 

Position: 

Office: 


2.  Please  note  that  your  participation  and  meaningful  responses  are  fundamental  to  the 
success  of  this  research.  No  thought  or  opinion  is  too  trivial  to  not  be  included.  What 
you  believe  is  an  insignificant  comment  might  trigger  a  response  in  one  of  the  other 
participants  in  the  next  round  of  questions. 


3.  Background:  prior  research  has  produced  a  decision  support  tool  that  relates  a  theater 
commander’s  mission  needs  with  combat  aircraft  (asset  sets)  and  their  associated 
deployment  footprints.  The  output  of  this  tool  is  shown  as  the  yellow  box  in  Figure  1. 
However,  integrating  the  presence  and  effect  of  campaign  specific  issues  can  improve  the 
tool.  The  goal  of  this  research  is  to  identify  and  develop  quantitative  measures  for 
campaign  specific  issues.  Figure  1  presents  a  hierarchy  of  campaign  specific  issues, 
previously  developed  through  discussion  and  interviews  with  you. 

C am paign  Issues 


Best  Value  Asset  Set 


2E 


[campaign  Specific  C 


Goodness  i  Generic  Mission  Goodness 


Figure  1 :  Campaign  Specific  Issues  Hierarchy. 
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CAMPAIGN  SPECIFIC  ISSUES  QUESTIONNAIRE 

ROUND  1 


CAMPAIGN  SPECIFIC  ISSUES  VERIFICATION  AND  PRIORITIZATION 

Please  refer  to  the  Campaign  Issue  Hierarchy  depicted  in  Figure  1.  To  help  you 
understand  how  each  issue  is  defined,  we’ve  included  a  “Definitions”  email  attachment 
that  provides  a  description  of  each  issue.  Each  issue’ s  definition  states  what  the  issue 
covers,  as  well  as  how  the  issue  is  quantitatively  measured. 

The  top-level  box  is  labeled  “Best  Value  Asset  Set.”  Possible  sets  of  fighter  and 
bomber  aircraft  will  be  analyzed  with  respect  to  both  campaign  specific  issues  and 
technical  capabilities.  Campaign  specific  issues  (green  solid  line  boxes)  and  Campaign 
specific  constraints  (red  double  line  boxes)  are  evaluated  under  the  “Campaign  Specific 
Goodness".  Depending  upon  the  particular  set  of  combat  aircraft  assets,  the  measures  can 
display  a  range  of  value  or  “worth”  to  the  theater  commander,  while  the  constraints  either 
permit  or  deny  the  use  of  a  particular  asset.  The  technical  capabilities  of  each  combat 
aircraft  with  respect  to  aerospace  missions  are  evalnated  under  the  “Generic 
Mission  Goodness”  measure  (yellow  dashed  line  box).  This  measure  is  outside  the 
realm  of  this  study.  Based  upon  the  information  obtained  from  this  decision  support 
tool,  campaign  planners  and  theater  commander  can  make  more  informed  decisions  in  a 
shorter  period  of  time. 

There  are  two  purposes  or  sections  to  this  phase  of  the  Delphi  study: 

-  Confirm  the  campaign  specific  issues  hierarchy  (Question  A) 

-  Order  the  issues  according  to  their  potential  influence  on  the  selection  of  aircraft 

assets.  (Question  B) 

QUESTION  A:  Based  upon  your  experience,  is  the  hierarchy  presented  in  Figure  1 
correct?  (Have  we  captured  the  right  issues,  in  the  right  respective  relationships?) 

PLEASE  ANSWER  ‘QUESTION  A’  HERE; 
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QUESTION  B:  Based  upon  your  experience,  please  rank  the  campaign  specific  issues  (1 
through  24,  where  1  is  most  important),  shown  in  Figure  1,  according  to  the  potential 
influence  their  individual  range  of  measurement  could  have  upon  the  selection  of  a  set  of 
combat  aircraft.  Each  measure  and  constraint  is  presented  in  bold  font.  Underneath  the 
measure  is  the  range  of  scores  to  which  a  unique  aircraft  asset  set  is  measured.  To  the 
right  is  an  underlined  blank  for  your  answer.  Please  type  your  numbered  responses  in  the 
blank  next  to  each  issue.  Please  refer  to  the  “Campaign  Specific  Issues  Definition” 
document  that  accompanied  this  survey  for  questions  on  issue  definitions. 

For  example,  if  you  believe  that  Climate,  the  effects  of  which  are  measured  as  0  - 
100  percent  mission  degradation,  is  the  most  potentially  influential  issue  in  selecting  a  set 
of  combat  aircraft,  you  would  place  a  ‘1’  next  to  Climate.  The  next  most  potentially 
influential  issue  will  receive  a  '2',  and  so  forth. 

CAA4PA1GN  SPECIFIC  ISSUE  AND  MEASUREMENT  RANK 

Multi-National  Compatibility: 

Compatibility  of  host-nation  and  US  aircraft  assets  (0-100  percent) 

Beddown  Location:  Fighter  Distance 

Pre-determined  staging  base  distance  to  enemy  target  (0  -  3000  miles) 

Beddown  Location:  Bomber  Distance 

Pre-determined  staging  base  distance  to  enemy  target  (0  -  6500  miles) 

Ability  to  Resupply 

Ratio  of  Consumption  to  Resupply  (0  to  1) 

Allow  Assets  In 
Yes  or  No  constraint 

Inter-national  Politics 

Yes  or  No  constraint 

Intra-national  Politics 

Yes  or  No  constraint 

Inter-service  Politics 

Yes  or  No  constraint 

Intra-service  Politics 

Yes  or  No  constraint 

Airfield:  Runway  Length 

Yes  or  No  constraint 

Airfield:  Weight  Capacity 

Yes  or  No  constraint 
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QUESTIONS  Continued: 

CAMPAIGN  SPECIFIC  ISSUE  AND  MEASUREMENT 

Ramp  Space 

Yes  or  No  constraint 

Munitions  Storage  Area:  Fighter  Munitions 

Percent  of  fighter  sorties  supported  per  day  (0-100  percent) 

Munitions  Storage  Area:  Bomber  Munitions 

Percent  of  bomber  sorties  supported  per  day  (0-100  percent) 

Petroleum,  Oil,  and  Lubricants  (POL) 

Percent  of  asset  sorties  supported  per  day  (0-100  percent) 

Tanker  Support  Required 

Tankers  required  per  day  (0-35  tankers) 

Topography 

Percent  degradation  of  mission  (0-100  percent) 

Climate 

Percent  degradation  of  mission  (0  -  100  percent) 

Asset  Mission  Flexibility 

Percent  capability  of  all  aerospace  missions  (0-100  percent) 

Asset  Set  Utilization 

Average  utility  of  all  aircraft  in  asset  set  (0-70  percent  average  utility) 

Asset  Set  Over-utilization 

Average  over-utility  of  all  aircraft  in  asset  set  (70  -  100  percent  average  utility) 
Availability  of  Asset 

Average  mission  capable  rate  of  aircraft  in  asset  set  (0-100  percent) 

Enemy  vs.  USAF  Capability 

Ratio  of  Enemy  to  Allied  Forces  (0  to  3) 

Enemy  Intelligence,  Surveillance,  and  Reconnaissance 

Percent  of  operations  exposure  to  enemy  (0-100  percent) 

Enemy  Interference 

Percent  degradation  of  mission  (0-100  percent) 

On-Going  Requirements 

Yes  or  No  constraint 


RANK 
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This  concludes  this  questionnaire.  As  you  return  the  questionnaire,  I  will  compile 
the  answers  given,  and  respond  to  you  with  a  second  round  to  the  study.  Within  this 
round,  any  comments  provided  by  you  within  Question  A  of  the  first  round  will  be 
included.  The  relative  rankings  of  each  campaign  specific  issue  will  be  presented 
according  to  the  consensus  or  average  of  rankings  received  by  you.  If  there  are  any 
extreme  differences  of  opinion  (i.e.  the  Climate  issue  was  ranked  second  by  one 
individual  and  twenty-second  by  another),  these  differences  will  be  addressed.  Thank 
you  very  much  for  your  time  and  participation  in  this  survey! 
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APPENDIX  G:  DELPHI  Study  Questionnaire  Three 

Campaign  Specific  Issues  Delphi  Questionnaire  Three 

Please  answer  all  questions  on  this  document  and  return  within  an  e-mail  to: 

Chrisotopher.Buzo@afit.  of.  mil 

The  Purpose  of  this  questionnaire  is  to  determine  relative  weightings: 

-  Between  the  sub-issues  within  a  parent  issue  (i.e.  Multi-National  Compatibility,  Allow 
Assets  In,  etc.  within  Host  Nation) 

-  Between  parent  issues  (i.e.  Host  Nation,  Enemy,  etc) 

During  previous  questionnaires,  the  relative  rankings  of  importance  between  the 
campaign  specific  issues  have  been  determined.  Based  upon  these  results,  issues 
recognized  to  be  more  potentially  influential  will  be  ranked  against  those  issues  not  as 
potentially  influential  to  the  selection  of  combat  aircraft  to  deploy  in  response  to  a 
contingency  crisis  in  the  Southwest  Asian  Theater. 

Please  compare  the  Campaign  Specific  Issue  in  ‘Column  A’  to  the  Campaign  Specific 
Issue  in  ‘Column  B’.  In  each  case,  the  issue  in  ‘Column  A’  has  been  identified  to  be 
more  potentially  important  than  the  issue  in  ‘Column  B’.  Looking  at  the  range  of 
measurement  for  each  issue,  PLEASE  STATE  HOW  MANY  TIMES  MORE 
IMPORTANT  THE  SWING  FROM  BEST  SCORE  TO  WORST  SCORE  FOR  THE 
ISSUE  IN  COLUMN  ‘A’  IS  THAN  THE  SWING  FROM  BEST  SCORE  TO  WORST 
SCORE  FOR  THE  ISSUE  IN  COLUMN  ‘B’. 

Consider  this  example.  Compare  Petroleum,  Oil  and  Lubricants  (POL)  to  Beddown 
Location;  Fighter  Distance.  The  swing  from  best  to  worst  score  for  POL  is  100  percent 
to  0  percent  support  of  asset  sorties  per  day.  The  swing  from  best  to  worst  score  for 
Beddown  Location:  Fighter  Distance  is  Omi  to  3000mi  distance  from  the  pre-determined 
staging  base  to  the  enemy  target.  I  believe  that,  if  POL  were  to  drop  from  100  percent  to 
0  percent,  this  drop  would  be  4  times  worse  than  if  Beddown  Location:Fighter  Distance 
were  to  increase  from  0  mi  to  3000  mi  from  the  enemy  target.  As  my  answer,  I  put  4  in 
the  middle  block. 


COLUMN  A 

IS  HOW  TIMES  MORE 
IMPORTANT  THAN 

COLUMN  B 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

4 

Beddown  Location:  Fighter  Distance 

(Omi  to  3000mi  distance  to  enemy  target) 

Based  upon  this  example,  please  provide  your  answer  to  the  following  comparisons.  If 
you  feel  that  the  issue  in  ‘Column  A’  is  less  important  than  the  issue  in  ‘Column  B’, 
please  use  fractions  (1/2,  1/3,  etc.) 
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QUESTIONNAIRE  3  ANSWER  SHEET 


COLUMN  A 

IS  HOW  TIMES 
MORE  IMPORTANT 
THAN 

COLUMN  B 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

Beddown  Location:  Fighter  Distance 

(Omi  to  3000nii  distance  to  enemy  target) 

Enemy  Vs.  USAF  Capability 

(3  to  0  USAF  to  Enemy  Ratio) 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

Enemy  Vs.  USAF  Capability 

(3  to  0  USAF  to  Enemy  Ratio) 

Availability  of  Asset 

(100%  to  0%  average  mission  capable  rate  of 
aircraft) 

Availability  of  Asset 

(100%  to  0%  average  mission  capable  rate  of 
aircraft) 

Multi-National  CapabiUty 

(100%  to  0%  compatibility  of  host  nation  assets) 

Beddown  Location:  Fighter  Distance 

(Omi  to  3000mi  distance  to  enemy  target) 

Availability  of  Asset 

(100%  to  0%  average  mission  capable  rate  of 
aircraft) 

Beddown  Location:  Fighter  Distance 

(Omi  to  3000mi  distance  to  enemy  target) 

Beddown  Location:  Bomber  Distance 

(Omi  to  6500mi  distance  to  enemy  target) 

Beddown  Location:  Fighter  Distance 
(Omi  to  3000ml  distance  to  enemy  target) 

Ability  to  Resupply 

(1  to  0  ratio  of  consumption  to  resupply) 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

Munitions  Storage  Area:  Fighter 

(100%  to  0%  fighter  sorties  supported  per  day) 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

Munitions  Storage  Area:  Bomber 

(100%  to  0%  bomber  sorties  supported  per  day) 

Petroleum,  Oil,  Lubricants 

(100%  to  0%  asset  sets  supported  per  day) 

Tanker  Support  Required 
(0  to  35  tankers  required) 

Topography 

(lOOVo  to  0%  mission  degradation) 

Climate 

(100%  to  0%  mission  degradation) 

Availability  of  Asset 

(100%  to  0%  average  mission  capable  rate  of 
aircraft) 

Asset  Mission  Flexibility 

(100%  to  0%  capability  of  aU  aerospace  missions) 

Availability  of  Asset 

(100%  to  0%  average  mission  capable  rate  of 
aircraft) 

Asset  Set  Over-Utilization 

(0%  to  100%  utility  of  all  aircraft  in  asset  set) 

Enemy  Vs.  USAF  Capability 

(3  to  0  USAF  to  Enemy  Ratio) 

Enemy  Surveillance,  Intel,  and  Recon 

(0%  to  100%  operational  exposure  to  enemy) 

Enemy  Vs.  USAF  Capability 

(3  to  0  USAF  to  Enemy  Ratio) 

Enemy  Interference 

(0%  to  100%  mission  degradation) 

Topography 

(100%  to  0%  mission  degradation) 

Beddown  Location:  Fighter  Distance 

(Omi  to  3000mi  distance  to  enemy  target) 

Thank  you  very  much  for  your  support  of  this  research.  I  greatly  appreciate  the  time  you 
took  to  complete  these  questionnaires.  I  will  be  concluding  this  study  with  a  conclusion 
questionnaire  in  which  I  will  present  the  individual  Campaign  Specific  Issue  weights,  as 
they  have  been  determined  to  you.  This  concludes  this  third  questionnaire. 
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APPENDIX  H:  Interview  Notes 


INTERVIEW  NOTES:  ACC  SME  #3  and  SME  #8-  ACC/XOX  -  1330  30  Nov 

Historically,  give  me  the  platform,  and  I  will  decide  what  to  do  with  it. 

Need  to  get  them  to  think  in  terms  of  what  do  you  want  to  do? 

Give  example  -  on-going  issue 

Northern  Watch  requirements  new  Look  originally  at  F-15E’s;  only  have  so  many,  not 
really  available  to  go  any  where  else. 

So  are  there  other  weapons  system?  Yes  (gives  examples) 

Can  we  have  certain  assets  flying  in  circles  over  areas. . .  country  issues 

(S.S.)  -  Lead  to  political  issues 

SME:  There  are  both  political  and  risk  issues 
A  lot  higher  risk  of  enemy  threat 

Political  issues  are  broad 

May  be  forward  basing  issues  Allies  allowing  bombers  to  be  based  in  country,  etc. 
Bombers  close  to  other  countries  (such  as  Russia  and  Chezchnia) 

Have  unintentional  consequences. 

(s.s)  Indirect  political  issues? 

SME;  Yes 

(a.j.)  Are  there  certain  considerations  along  inter-political  issues? 

SME:  Yes 

POM  questions  -  if  we  (The  Air  Force)  do  it  (Fight  the  battle)  and  continue  to  do  it,  we 
get  bigger  slice  of  the  pie.  Upcoming  QDM  is  an  issue 

Classification  of  LD/HD  (low  density/high  demand)  assets,  such  as  the  A-10 
Navy  may  allow  A-10  use  over  F-18  (applicability  of  use) 

(S.S.)  So  there  are  direct  and  indirect  and  cross-service  (political  considerations)? 

SME:  (That’s  a)  Good  way  (of  putting  it) 

(S.S.)  (What  about)  the  favoritism  of  planning  staff 

SME:  (That  is  a)  Factor,  but  don’t  know  howto  quantify 
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The  Guy  who  writes  the  thing  (the  plan/Request  for  forces)  for  the  CfN-C,  that’s  the  guy 
with  the  power. 

This  is  what  I  want,  If  don’t  have  the  component  in  planning,  it  doesn’t  get  there 
You  want  it  on  paper. . . what  is  on  paper  goes 
Want  to  work  like  hell  to  get  right  stuff  on  paper 

Well,  I  need  a  battlegroup,  but  do  I  need  subs?  No,  Frigats?  No.  It  (the  battlegroup) 
could  be  tailored  and  can  still  keep  others  flowing  on  to  different  areas 

(S.S.)  Basing  Considerations. . . . 

SME;  Tanker  support  is  an  issue.  IT  was  a  big  issue  in  Kosovo.  We  didn’t  have  enough 
Limits  how  far  we  can  go 

Will  they  have  to  commute  to  the  war?  We  opt  for  one  asset  over  another  because  it  can 
go  farther. 

Who’s  the  opposing  force?  Gives  threat  characteristics 
Anticipated  AOR  and  beddown  characteristics 
Political  considerations  where  we  can  operate 

Kingdom  (Saudi  Arabia)  not  wanting  offensive  operations  from  their  area 

Most  of  Kuwait  already  at  the  MOG  (Maximum  aircraft  on-ground),  so  we  have  to  go  to 

other  areas 

Infrastructure. . .  how  we  handle  the  aircraft 

Air  refueling.  Munitions.  CALCMS  (conventional  air-launched  cruise  missiles) 

SME;  Risk  is  a  political  considerations,  it’s  dynamic.  Changes  as  the  campaign  moves  on 

Planners  don’t  often  know  the  capabilities  of  (our)  assets 

Lack  of  knowledge,  lack  of  confidence  is  big  (blue  and  purple  suitors) 

We  default  to  the  asset  or  weapon  that  we  know  works 

Planners  want  what  they  know  until  they  see  the  success  of  another,  such  as  the  JDAM, 
then  we  push  to  that 

Until  that  happens  (knowledge  and  confidence)  occurs,  it  won’t  happen 


MAJOR  JOHNSON’S  NOTES  (at  same  interview); 

DCAPES  - 

Ops  to  log  link  -  Level  4  detail  plans 
AF  component.  Follow-on  to  COMPES 

JOPES  -  gives  level  3  detail  now 
Part  of  COMPES  (deliberate  and  crisis  action  planning) 

CON  PLAN  “Lower  scale”  0-plan? 

Issue  -  Blue  suit  planners  on  unified  command  staffs  now  ask  for  capabilities  (i.e.  15 
squadron)  vs.  effects. . .  i.e.  want  a  power  grid  taken  off-line  for  x  hours 
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Chris’  Question  “Factors” 

-  Political  sensitivity  -  (direct)  forward  basing  (what  on  ally  may  permit) 

-  Risk  -  Loss  of  asset  -  external  (indirect)  percieved  threat  (Russia  may  worry) 

Loss  of  life/collateral  damage 

-  (Internal)  cross-service  rivalry  (POM  issues-funding) 

-  Mil  issues  (Tanker  support?) 

~  Range,  payload 

-  Enemy  sophistication  (F-16  radar  sufficient?) 

-  Beddown  considerations  (infrastructure) 

-  Lack  of  knowledge/confidence  -  mission  planners  selecting  innapropriate  asset 
~  munitions  become  the  “flavor  of  the  month” 

F-15E  AGM-130  -  typically  only  8  aircrews/sqdrn  -  not  trained 

-  precision  stand  off  weapon 

Split  ops  issue  -  spare  parts  are  critical! 

-  4  Squadrons  at  Shaw 

~  Two  independent,  1  maybe  independent,  1  not  independent 
~  Hence  can  only  deploy  2  seperately 

-  Split  ops  is  very  expensive 


End  of  formal  interview. 
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INTERVIEW  NOTES:  ACC  SME#9  -  ACC/XOX  -  1430,  30  Nov  99 


(Interview  introductions,  background,  and  reason  for  visit) 

It  sounds  like  you  want  to  talk  to  the  JOINT  DEPLOYMENT  PLANNING  OFFICE, 
C2TIG,  at  Hurburt  Field. .  .they  belong  to  the  C2  Center 
AC2SR  center  .  A3/A4,  look  at  the  AOC 

Use  example  of  Allied  Force  Noble  Anvil 
F-16  C/Js  are  optimized  for  SEAD 

Efficiencies  gained  when  combined  with  other  missions. .  .HARM  pods 

Of  course,  there  are  limits  on  PODS,  kits,  training  kits,  testing  kits,  skilled  maintenance 

technicians,  munitions  troops  who  are  smart  on  the  HARM 

Re-address  how  we  have  major  weapons  systems  laid  down 
Open  up  FOLs,  pick  up  and  move,  what’s  to  stop  us  from  doing  so? 

Do  we  have  weapons?  Inter-theater  movement 
Increases  tanker  requirements 

Test  equipment  may  or  may  not  be  there,  we’re  limited  on  equipment 
We  need  to  know  when  they  are  operational 
What  about  the  airlift  needed? 

Barebase  operations  -  what  is  the  minimum  time  we  can  begin  to  execute  (plan)  and 
when  are  we  ready  to  operate? 

What  needs  to  come  in?  Setup  Prime  BEEF,  base  operations,  etc. 

We  need  a  communications  backbone 
POL,  munitions 

Look  at  combat  planning,  what  would  opening  Turkey  up  do  (for  us)? 

It  gives  us  access  to  the  area.  We  want  to  shape  the  battlespace! 

Aerospace  superiority  is  important,  but  getting  there  first  is  more  (important) 

Put  C2ISR  in  (to  the  battlespace)  at  less  risk,  get  a  better  picture 

Combine  this  with  overhead  and  ground  (C2ISR),  and  we  have  a  better  picture  (of  what’s 
going  on) 

We  also  want  to  mask  our  ability  for  the  enemy  to  see  what  we’re  doing. 

We  want  to  look  at  the  threat,  our  enemy’s  capabilities 

We  want  to  protect  against  our  enemy  finding  out  what  we’re  doing 

Of  course,  there  are  ways  to  protect  against  it. . .  .but  then  there’s  the  “CNN”  effect 

Sometimes  we  want  to  let  the  enemy  know  that  something  is  coming 

We  bring  in  C2ISR  to  make  public  what  the  enemy  is  doing  (bad  things) 

Where  we  put  our  assets  may  have  diplomatic  impacts,  and  sends  message  (to  enemy), 

gives  us  a  deterrent  effect 

We  bring  in  assets  for  deterrence  or  for  execution 

The  timing  of  where  and  when  to  bring  it  in  is  also  important 
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(S.S.)  Would  you  agree  that  politics  may  affect  what  we  bring  in? 

SME:  Agree  that  political  situations  may  affect  (what  we  bring  in) 

Strict  ROEs  have  impact. . .  Stand-off  munitions  and  precision  guidance  munitions 

The  CINC  brings  in  rules,  we  have  to  go  with  them 

Certain  aircraft  and  configurations  may  be  driven 

We  want  to  go  to  effects  based  planning,  as  we  move  to  AEF  construct 

On-Call  AEW  -  12/12/12/6 

We  know  the  capabilities  it  brings,  plus  x  number  of  LD/HD  assets 
If  we  plan  for  ahead  of  time  what  you  can  get,  it  simplifies  planning 
We  then  start  to  base  it  on  effects  based  planning 

(S.S.)  Munitions? 

SME:...pre-positioned  both  sea  and  land 

Moving  munitions  by  air  is  not  efficient  or  cost-effective 

We  may  bring  them  in  at  first  this  way,  but  sustainment  isn’t  good. 

(S.S.)  Political  Constraints? 

Most  political  constraints  deal  with  the  sensitivities  in  particular  country.  What  they’re 
willing  to  accept. 

Offensive  capabilities?  Maybe  not. 

Defensive  capabilities?  Maybe.  ISR?  Maybe 

Of  course,  we  can  work  around  that,  that’s  the  purpose  of  the  secstate  (secretary  of  state) 
It  may  have  timing  impacts,  with  negotiations 
The  planning  must  allow  for  negotiations 

For  a  given  CO  A,  here’s  my  optimum  set,  once  agreed  upon  it  may  not  be  optimal 

Site  surveys,  they  may  not  be  up  to  date.  As  well  as  base  support  plans.  We’re  now 
looking  at  expeditionary  support  plans 

The  C2  center  is  responsible  for  EFX,  they  would  be  good  people  to  talk  to. 

Getting  folks  over  there  -  airspace  restrictions  enroute.  Depends  on  the  situation,  the 
time.. affects  our  log  flow 

Setting  up  FOLs;  clearances  to  and  from  the  FOL  and  Battlespace  procedures 
Combat  airspace  control  is  very  important!  We  need  to  know  what  is  going  on  in  three 
dimensions...  not  just  1.  What’s  coming  in,  what’s  going  out.  Commercial  non-players 

(SME;  Questions  interviewer) 

So  what  will  this  run  off  of?  What  does  it  come  from?  What’s  your  bandwidth?  You 
want  a  tie  in  with  existing  data  stores...  Global  Command  and  Control  Structure. 

Focus  on  Just  In  Time  or  Agile  Logistics.  Can  we  do  it?  This  is  a  major  paradigm  shift! 
For  both  logistics  flow  and  combat  planning 


135 


MAJOR  JOHNSON’S  NOTES  (at  same  interview): 

C2TIG  group  at  Hurlburt  field. . .checkout! 

Make  sure  the  A3/A4  link  is  there  without  solely  focusing  on  J3/J4 
AF  may  not  always  be  the  JFACCI 

“Factors” 

F-16  CJ  is  optimized  for  SEAD  -  need  the  HARM  pod  to  make  the  system  work 
Split  ops,  bare  base  ops  issues  (infrastructure) 

MOG 

Tactics  issues  (battlespace  shaping) 

Multiple  beddown  sites  add  axes  of  attack 

Information  warfare/security  and  OPSEC  -  can  also  do  in  reverse. . .  scare  em  with  the 
CNN  effect 

Some  locations/assets  selections  more  susceptible  to  info  flier  to  enemy 
Surveillance  assets  (AW ACS,  RJ)  placement  “we’re  watching  you 

Want  to  transform  CINCs  from  asking  for  specific  assets  to  specifying  requirements 
(desired  effects) 

End  Of  Interview 
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INTERVIEW  NOTES:  ACC  SME  #10  -  ACC/XOX  -  30  Nov  99 


Joint  Publications  is  a  good  start. 

Intangibles  make  campaign  planning  an  art  not  so  much  as  a  science... may  never  be 
Revolves  around  individual  culture  and  purgatives  of  host  nation 
Beddown  of  aircraft  in  Turkey  (Kosovo)  totally  driven  by  Turkish  politics 
Where  wanted  to  send  signals  to  own  people,  other  countries 
Always  going  to  be  a  coalition 
We  (The  United  States)  bring  $  into  the  infrastructure 
Coalition  politics  vs.  Host  Nation  desires 
Important  of  These  will  vary  as  far  as  nation 
Want  to  look  at  In-Theater  CINC 
Allied  Impact 
CINCs  campaign  planning 
Inpact  to  Enemy  -  actions  of  enemy 
Will  bring  different  things  if  different 
Enemy  Intentions/  Enemy  Actions 

Often  plan  for  most  dangerous  and/or  most  likely  enemy  action  when  thransfer  Oplan  to 
Opord,  want  to  make  sure  we  know  what  the  enemy  is  doing. 

Not  only  what  enemy  is  doing,  but  CINCs  comfort  zone  of  risk  of  enemy  actions 
Mission  accomplishment  vs.  force  survival  -  Tactical  vs.  Strategic 
Leads  CINCs  to  ask  for  more  and/or  different  forces  than  if  different  case 
Beddown  is  based  on  Enemy  Actions  and  political  constraints 
Will  host  nations  allow  (us  there)  and  allow  us  to  fly 
ROEs  and  host  nation  support 
Country  approval/permission/support/sensitivities 
State  Department,  Ambassador,  handle  this,  all  at  CINC  level 
Goldwater-Nichols  act  -  all  forces  assigned  to  specific  CINC,  JFOR  document 

Host  Nation  politics 

Nations  may  allow  defensive  operations,  but  not  offensive 
Nations  may  allow  fighters,  but  not  bombers 
Nations  may  allow  Stealth,  may  not 

Nations  may  allow  only  certain  countries  (NATO/UN)  to  enter  their  land 
National  interest  in  the  location  of  basing  assets 

US  Domestic  politics  can  be  summed  up  in  force  protection. . .  commander  desire  to  use 
specific  asset 

-AT  THIS  POINT,  HIERARCHY  WAS  INTRODUCED  AS  INTERVIEWEE  ASKED 
“WHAT  ELSE?”  (Several  times)— 

Airfield.... 

Approach  considerations 
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Big  since  Senator  Brown  Crash  (gives  example  of  TACAN  Dept  Defense  Certifications. 
Must  be  certified  to  allow  US  pilots  to  land,  not  all  UN  country’s  certifications  up  to  par 
US  Turps  Criteria 

Survivability  Considerations:  Terrain/Obstructions 

Host  Nation  Airfield;  do  they  have  approach  considerations?  Is  host  nation  Day  View 
Clear  (Visual  Approach)  Only? 

Weight  Capability  -  Yes,  Absolutely.  Kosovo  example 

MSA  -  Yes.  Port  On  &  Off  -  Loading  capabilities,  capability  of  transfer  from  port  to 

Base. . .  important  for  flow 

Again,  Host  nation  issues  are  a  key 

Don’t  have  a  lot  of  pre-positioned  flow  of  munitions 

Weight  bearing  limits  of  roads 

Munitions  storage  capability  -  do  they  meet  standards? 

Non-NATO  bunkers  may  be  iffy  to  secure  munitions 
Goes  into  survivability,  where  beddown  aircraft 
POL  -  Critical  issue 

NATO  has  POL  infrastructure  built  in,  not  real  problem 
Everyone  flies  JP-8 

ClimateAVeather  -  Certainly.  “Do  different  aircraft  react  differently  to  varying  weather 
conditions?”  No. 

Rating  of  approach,  pilot  experience  is  key. 

FAA  rules  don’t  apply,  follow  Air  Force,  which  are  more  stringent 

Ability  to  Crossover  -  Interrelated  to  all  attributes  of  hierarchy 
Ramp  space.  Sortie  Generation 

ACC/XOX  office,  might  know  WMP 

CENTAF  A5  -  Contingency  Planning  at  SHAW  AFB 
A3/DOO...DSN  965-2966 


MAJOR  JOHNSON’S  NOTES  (at  same  interview); 

Process  works  pretty  much  as  the  books  say  -  Joint  pub  1 
Intangibles  suggest  that  campaign  planning  is  more  an  art  than  a  science 

Beddown  -  driven  largely  by  host  nation  -  their  national  interests 
~  Turks  gave  NATO  3  beddown  base  options 

—  Draws  infrastructure  development  (money,  systems  into  local  economy,  jobs) 

—  Their  actions  suggest  intentions  toward  their  enemies/neighbors 
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Coalition  Operations  -  nothing  done  in  a  vacuum  anymore 

Must  have  a  way  of  measuring  enemy  intentions  and  actions 

CINCs  comfort  with  understanding,  risk  of  enemy  actions 
—  Friendly  force  survival  vs.  mission  accomplishment 

Talk  to  “POL  MIL”  advisors  at  a  Unified  Command 

Domestic  Politics 

Joint  Beddown  issues 
Can  work  against  us 

Not  trusting  replacement  parts  offered  by  host  nation  -  thus  potentially  losies  sorties 
and  hurting  our  relationship  with  the  host  nation 

Airfield  issues;  approved  takeoff/landing  approaches  at  host  nation  beddown  sites 

Port  throughput,  roadways  (for  delivering  support  and  munitions) 

Approved  (certified)  for  offloading  munitions 
Weight  bearing  limits 
Munitions  storage  availability 

WMP-5  (War  and  Mobilization  Plan)  —  Classified  Secret 
Gives  sortie  durations,  turn  times  (sortie  generation  ratios) 

Used  mostly  for  planning,  funding  purposes 


END  OF  INTERVIEW 
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INTERVIEW  NOTES:  ACC  SME  #6  and  SME  #7  -  ACC/XOX  -  1  Dec  99 


ACC  Reacts  to  whims  of  CINCs 

Mission  Capable  rates  of  the  aircraft 
What  is  best  unit  to  select,  suitable  subs  if  non  are  good 
HARMS  target  system,  Pods,  Test  equipment  availability 
By  time  received  order,  force  mix  already  determine;  just  choice  of  selecting 
people/places  to  fill 

One  controversy  over  fijse  for  B-2  ID  AM 
Current  version  sufficient,  preferred  to  use  new  uncertified  version 
By  the  time  we  receive  execute  order,  already  a  done  deal 

Lot  of  concern  over  munitions. .  .IDAM,  CALCM.  Seems  didn’t  put  much  forethought 
into  mixes,  allocations 

Need  to  think  what  do  CINCs  need,  not  just  pick  and  go 

If  you  want  CALCM,  here’s  what  we  have,  here’s  how  many  we  have  allocated  to  other 
places 

Started  off  with  CALCMs  early,  stopped  later  on 
Think  Later  on,  instead  of  using  all  first  off;  use  more  smartly 
HGS  Pods  systems  (HARM)  -  emptied  out  Shaw  to  keep  Kosovo  going 

(SME  #6  inserts...) 

Alert  forces  -  2nd  plan  to  send  more  people  in 

Size  of  Augmentation  force  depended  upon  COA 

CINC  now  asks  for  LD/HD  first  -  want  to  see  everything  at  all  times 

(Maj  S.S.)  Do  any  factor’s  change? 

(SME#7)  Beddown,  Target  Sets  -  NATO  determined  TSs,  delayed  ATO  generation 
(SME#6)  Turkey  only  allowed  Mil- Air  in 
LBM  -  Load  Bearing  Capability  of  runway. .  .became  an  issue 
Looking  at  both  what  and  where  to  beddown 

Turkey  has  strict  rules;  tried  to  bring  beddown  equipment  in  on  747’s 
High  LBM  (78-80  psi)  -  Turkish  runways  couldn’t  handle  it 
(SME  #6)  Sometimes  they  make  you  do  that  -  “you  can  land  here,  but  you’ll  pay  for  it” 
Looked  like  Turkey  held  off  initially  (on  letting  us  beddown) 

Most  forces  (for  Kosovo)  were  in  Italy  and  England,  no  big  culture  issues 
Trucking  and  rail  can  be  an  issue 

Can’t  always  “FedEX”  -  Limits  on  what  can  bring  in  commercially 

(Maj  S.S.)  Any  issues  like  national  government’s  required  who  does  stuff?  (i.e.  G.I.s) 

We  try  to  minimize  footprint,  encourages  host  nations  to  provide  support 
USAFE  could  talk  Host  Nation  (better  than  us) 

MSgt  Caldwell  -  Contracting  NCO,  worked  Ramstein  Issues  (ext  5096) 
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If  system  can  tap  into  current  information,  people  can  see  what’s  available  and  tailor 
from  there  -instead  of  sending  entire  UTC,  bring  an  aircraft 
(We’re)  in  the  process  of  breaking  down  UTCs  from  18-24  ship  to  6  ship 
Agile  Combat  Support  -  Setting  up  EAF  center  for  1  stop  shop  POC  for  info 
ID  types/quantities  of  equipment  needed 
Want  to  have  recommended  cargo  deployment  listings 

Unless  you  have  the  entire  support  operation  at  the  Forward  Operating  Location  (FOL), 
Can’t  do  split-ops. . .Talk  to  ACC/XOD 
Spares,  Test-equipment,  and  troops  are  an  issue 
Cross-training  of  pilots  in  multi-role  (missions)  is  an  issue 


MAJOR  JOHNSON’S  NOTES  (at  same  interview): 

Log  planner  -  experience  in  Kosovo  battle  planning  staff 
Force  mix  already  identified  -  his  job  partly  to  find  the  best  units  to  fill 

Mission  capable  Rates  -  what  units  best  to  send  (for  given  capability) 

Concern  on  munitions 

Operators  wanted  a  particular  JDAM  fuse  -  hadn’t  passed  OT&E  though 
Not  much  forethought  on  munitions  planning 
CALCM  example  (they  were  probably  overused  initially) 

LD/HD  issues  -  everyone  wants  them 

PAX  mostly  restricted  to  mil  air 

Restrictions  on  commercial  shippers  -  Turkey  only  allows  DHL 
(SME  #6) 

Agile  Combat  Support  -stand  up  an  AEF  center  -  so  deploying  units  have  a  single  point 
to  call  for  logistics  issues  (AGE,  etc) 

Improve  comm,  asset  visibility 

Deploying  units  then  won’t  have  to  take  an  entire  UTC 

Split  Ops  -  6  of  24  deploying  F-15s  would  need  75%  {given  estimate,  not  hard fact)  of 
the  squadron’s  support  -  won’t  work! 

Split  ops  briefing  (ACC/XO)  shows  costs  for  people,  support  equip  in  a  Northern  Watch 
issues 


END  OF  INTERVIEW 


141 


INTERVIEW  NOTES:  ACC  SME  #5  -  ACC/XOX  -  0855  1  Dec  99 


Interview  Introduction  and  summary  by  Lt  Buzo 

Biggest  (issue)  is  human  -  “I  want”  vs.  practical 
ACC  sources/tasks/flows 
Look  at  LIMFACS  (Limiting  Factors) 

We  need  a  leader  (in  a  crisis)  who  is  willing  to  make  decisions 
We  get  into  decision  loops  -  leads  to  delays,  indecisiveness 
A  good  plan  today  is  better  than  a  perfect  plan  next  week  -  Gen  Patton 
Need  to  be  carefiil  that  we  don’t  fall  into  this  trap  of  waiting  forever 
In  perspective  -  Deployment  orders 
UCOMM  -  planning  incrementalized  air  war  -  USAFE  Lead 

Began  to  develop  plans;  what  are  weapons  we  need,  then  look  at  weapons  systems  to 
deploy  these  weapons 

Human  interjection  comes  in  “I  want  this  aircraft”  (Human  Factor) 

Really,  what  is  capability. .  .capability  vs.  weapons  systems,  we  don’t  have  unlimited 
assets 

Global  Military  Force  Policy 
Tends  toward  AF  -  HD/LD  assets,  monitors  use 
F-1 17,  B-2,  not  under  GMFP,  but  don’t  have  that  many 
Look  in  terms  of  capability  vs.  Desired  effect 
2  Problems. . . 

Lock  ourselves  into  plan  we  can’t  live  up  to 
Competing  priorities  of  aircraft 

Availability  of  weapons  is  another  important  issue. .  .CALCMS 

USAFE  makes  Christmas  list  of  needs 
1  St  source  in-theater,  shows  short-falls,  send  up  to  UCOM  for  validation 
Sanity  check,  UCOM  sends  Request  For  Forces  up  to  JCS 
GMFP  -  Steady  state  (everyday).  Surge  (costs  in  air  frame,  personnel,  up  to  60  days  in 
length,  then  reconstitute),  No-Kidding  all  out  war  (Everything  out,  not  very  long  if  at  all) 

For  some  aircraft  (U-2)  pilots  are  strong  LIMFAC,  becoming  worse 

JS  does  scrub  on  GMFP  -  says  you  can  have  X 
Goes  to  JFCOM,  source  at  ACC 
ACC  goes. . .  you  can  have  X,  need  to  wait  on  Y 
Invites  problems  on  human  side 
Would  be  better  to  request  capabilities 
ACC  sends  back  comments  -  JC  comes  back  with  deployment  order 
With  technology  (e-mail)  almost  a  bartering  system 
Produces  proliferation  of  orders  -  get  new  before  can  adequately  CHOP  previous 
Need  to  see  to  Joint  community  a  change  in  doctrine 
Joint  Forces  Command  J33 
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Advocates  for  Joint  World 


TRANSCOM 

Who  controls  Assets/P ACOM/EUCOM/CENTCOM  in  and  out  of  AOR  is  important 
OPCON  should  transfer  in  and  out  of  AOR 

Bombers  -  Jstaff  thinks  you  may  have  so  many  in  reserve,  really  don’t  (B-2  ex) 

Want  2,  think  have  18  in  reserve,  but  don’t 

GEO  Political  issues 

Italy  doesn’t  like  B-52s. .  keep  in  air,  requires  more  tanker  support  than  if  could  have  in 
Italy 

Offensive  vs.  Defensive  capability 
B-2s  -  basing  is  an  extreme  problem 
Money  does  drive  decision,  can’t  advocate  it 
Doesn’t  care  about  the  dollar,  but  generals  do 

INTER/INTRA  political  issues 
B-2  pushed  into  Kosovo,  F-1 17  same 

Human  factor  -  humans  have  to  make  decision 
Need  to  convince  leaders  to  be  leaders  and  make  decisions 

Talk  to  WHITMAN  folks  who  handled  Kosovo  war  in  Conus 
Issues  they  have  on  factors  “from  home” 


MAJOR  JOHNSON’S  NOTES  (at  same  interview): 

Human  issues  (what  I  want,  vs.  what  I  need) 

Concerned  that  more  choice  will  exaggerate  indecisiveness 
Decision  loops  get  too  long  (good  early  decisions  usually  better  than  the  best  late 
decision) 

“I  need  F-1 17s  because  they’re  cool”  -  we  think  of  what  we  want,  vs.  what  we  need 
Need  to  express  needs  in  terms  of  desired  effects 
Capability  vs.  MDS  (requirement  for) 

Global  Military  Force  Policy  (GMFP) 

SECDEF  program  that  monitors  LD/HD  assets 

Email  speed  causes  too  much  “what  iflfing” 

Lose  track  of  what  the  real  deployment  order  is,  for  example 

Geopolitical  issues 

Italy,  Spain  don’t  mind  fighters,  but  hate  B-52’s  (nuke  stink) 

Kossovo. . .  good  to  loiter  B-52s  with  CALCMs. . .  if  fly  from  Fairford,  will  need  air 
refueling,  but  if  could  fly  from  Italy,  would  not  need  air  refuel) 
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Cost  -  “military  should  not  worry  about  costs. .  .but  the  generals  do” 

B-2’s  would  be  costly  to  deploy  (basing). .  .”a  fussy  aircraft” 

B-2’s  are  being  used,  partially  to  justify  its  existence 

Suggests  we  talk  to  Whiteman  AFB  on  their  CONUS  support  of  Kosovo  (B-2’s) 
Either  wing  cc  or  ops  commander 
What  worked  or  didn’t  work 
Do  it  fast,  before  expertise  melts  away 


END  OF  INTERVIEW 
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INTERVIEW  NOTES:  ACC/SME#11  -  USAFE  -  1  Dec  1210 


Inter-service  political  pressure 
Army  is  usually  the  JTF 

Requirements  for  collateral  damage,  casualty  minimization  (0) 

Achieving  (trying  to)  psychological  effect 
Knowledge  of  airpower,  systems 

Political  considerations  affect  target  sets  from  get-go 
Feedback  of  what  we  have  vs.  what  we  want  to  target 

National  support/improvement  of  infrastructure  we  bring 
Logistics  tail  -  C-5  break  rates,  can’t  always  depend  on  when  things/supplies  coming  in 
Survey  analysis/site  surveys  need  to  be  consolidated  and  improved 


MAJOR  JOHNSON’S  NOTES  (at  same  interview): 

Doctrine  -  what  service  calls  the  shots? 

Pace  -  “gradualism”  used  in  Kosovo 
How  fast/scale  is  the  target  list  addressed? 

Capabilities  affect  target  chosen 

As  bad  as  JOPES  is,  its  light  years  ahead  of  other  nations 
Beddown  data  collection  needs  to  be  improved 
AMC  collects  only  what  they’re  interested  in 

Lots  of  surveys  done  now,  but  data  integration/dissemination  must  be  done  better 


END  OF  INTERVIEW 


INTERVIEW  NOTES:  HQ/USAF  SME  #2  -  AF/XOPX  -  7  Dec  99 

EAF  concept  -  lead  to  requirements-based  force 
“what  do  you  want. .  .we’ll  give  it  to  you” 

Fighter  community  forward  operations  -  not  conformed  to  hand  purpose  of  EAF 
Doesn’t  really  affect  LD/HD  -  doesn’t  change  their  schedules 
Overflight  is  political  @  a  high  level  -  not  really  constrained  to  specific  aircraft 
Could  be  country  wanting  specific  aircraft 
Pilots  limit  capability 

ISR  -  UAV’s  Army  information  obtained  may  not  get  to  AF 

Historically,  look  at  CINCs  -  look  at  how  reqt’s  changed  as  color  of  suit  changes. . .  who 
the  CINC  is 

B-2/Stealth  aircraft  in  Kosovo  -  going  to  use  it 
Life  Span/Cost  -  Risk/Reliability  of  aircraft 

A- 10  extended  -  wanted  to  transfer  to  Army  -  Army’s  Air  Force  weapon  of  choice 


END  OF  INTERVIEW 
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INTERVIEW  NOTES:  HQ/USAF  SME  #3  -  AF/XOOC  -  8  DEC  99 


Maintainability  of  Aircraft 

Fuel  and  Munitions  -  be  there  or  not. . .  don’t  have  instantly 
Basing  considerations. .  .Beddown  location.  Distance  to  Target 
Affects  refixeling  considerations 
Enemy  Capabilities 

Enemy  Air  Defense  Simulator  is  a  good  tool 
Risk! 

Force  closure  -  real  time  Ops  and  logistics 

Introduced  into  interview:  HQAJSAF  SME  #  6  and  SME  #  7 

Type  of  munition  you  have 
DMPE  -  within  target  array 

PK  requirements  -  drives  #  of  airframes  to  weaponeering 

RISK!  -  stealth  of  standoff  if  risk  is  high  enough 

Other  mission  requirement;  may  require  CAS  or  SEAD,  etc. 

Used  everything  else...  OPS  TEMPO  of  specific  assets 
Weather  drives  where  locations  are  located  (overall  weather,  not  day-to-day) 
Commander’s  desire 

Speak  with  CENTAF  -  9th  AF. 

Look  at  the  Capabilities  Based  Logistics  Plan  and  Operations  Planner 
Software  decision  support  tools  that  succeed  JPT 

If  you  can  start  with  a  set  of  MRRs,  only  bring  what  you  need 
Air  staff  is  developing  3, 6, and  9  ship  UTCs  build-up  from  there  if  needed. 

END  OF  INTERVIEW 
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INTERVIEW  NOTES:  HQ/USAF  SME  #8  and  SME  #9  -  AF/XOPW  -  8  Dec  99 


Theater  CINC  delineates  capabilities  and  PGM,  OCA/DCA 
Still  sometimes  ask  for  specific  equipment 
Look  at  AEFs  in  UTC  construct,  pick  out  ULNs  into  TPFDD 
Identify  specific  forces 

(SME  #9)  TPFDD  libraries  -  10  AEF,  2  AEWs,  and  1  “Enabler” 
go  into  these  buckets,  depending  upon  locations,  and  look  for  capabilities  in 
buckets. . .  peel  out  what’s  needed 

Contingency  portion  of  AEF  forces  are  used  in  steady  state  -  look  in  units  to  see  if  you 
have  enough  forces 

(look  at  SORT  S/Persons/Tasking  elsewhere)  If  no,  look  to  AEW 
Capability  full  of  UTCs 

Go  down  AEF  list  if  don’t  have  required  number  of  assets 
On  issues  - 

Beddown  -  if  can’t  get  beddown  close  to  fight,  go  to  different  asset 
Cross  section  -  Aircraft  size/stealth 
Sometimes  political,  sometimes  site  survey 
Availability  of  munitions  is  a  factor 

Tasked  for  MTW?  Doesn’t  want  to  pull  asset  fi-om  another  event 
Status  of  aircraft 

Some  assumptions  made  -  weather,  staging  locations,  some  of  ROEs  could  come  up 
Political  issues  can  come  up  front,  first 

All  deliniated  in  planning  assumptions,  often  assume  away  issues  (such  as  weather) 
Weather  should  be  expected 
Support  comes  with  aircraft 
SVS,  Weather,  Comm,  etc.,  .meat  of  deployment 

“What  do  you  need  to  do?” 

CINC  asks  for  capability 

Might  be  service  politics  to  choose  with  aircraft  to  use 
(SME  #9)  wouldn’t  use  as  factor 

Enemy  capabilities?  Yes 
Air  defense  capabilities?  Stand-off 
Ground  protection 

Possible  reason  why  B-2  deployed  from  CONUS 
Look  at  AF  10-400,  AEF  instruction 
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SITE  Surveys 

Ground  capable  of  grining  in  aircraft?  Ramp  space? 

Fuel/  Fuels  source  considerations 

Beddown  -  CONOPs 

Might  not  be  able  to  discern  political  concerns 
Commonality  of  maintenance 

Don’t  want  multiple  beddowns  of  one  aircraft  -  economies  of  scale 
May  be  assumption  that  we  get  host  nation  support  for  asset 
Host  Nation  Support  Agreement 

Capabilities  of  common  aircraft  among  different  nations 
Could  be  a  political  consideration 
Reserve  components?  Should  they  be  activated? 


END  OF  INTERVIEW 
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INTERVIEW  NOTES:  AU  SME  #3  -  JDACC  -  10  DEC  99 


Joint  Doctrine  Air  Campaign  Course  evolved  from  Checkmate 
Desert  Storm,  didn’t  have  a  great  way  to  plan 
Developed  from  Joint  Doctrine  3.56-1 
2  week-course  -  a  lot  of  AF,  but  also  other  3  services  go 
Go  through  5  stages  of  campaign  planning 
Use  air-power  in  most  effective  way 
Mostly  operator  oriented 
Confirm  what  the  air  power  can  do 
Concept  Validate  -  confirm  with  Centers  of  Gravity 
Not  really  Deliberate  Action  Planning 
Focuses  on  Process,  not  Product 

Realizes  there  are  factors  or  “constraints”  and  “restraints” ...  drive  of  class  is  to  get 
students  to  think  about  these 

Did  not  have  any  taught  or  presented  list  of  “constraints”  or  “restraints” 

Suggested  speaking  to  JAK2C  course  instructors  at  Hurlburt  field 
(DSN  487-4409)  JDACC  course 


Possible  issues 

Logistics  concerns  and  support 

Enemy’s  in-ability  to  fight  @  night. .  .want  to  capitalize  on  that 
Intra-service  issues  can  arise 

Staging  locations  -  countries  not  letting  them  fly  in  offensive  weapons 

You  have  to  talk-talk-talk  your  wasy  through  the  planning  process 
Communication  is  Key 

Could  be  on  your  own  -  don’t  depend  on  coalition  forces,  but  do  look  to  Joint  Services 
END  OF  INTERVIEW 
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INTERVIEW  NOTES:  AU  SME  #2  and  SME#4  -  CWPC 


Asked  if  I  had  studied  the  TARGET  analysis  tool. 

In  any  contingency,  have  deliberate  planning  process 
Want  to  understand  AOR. .  .ROE,  assumptions,  should  initially  be  thought  about 
Look  @  5  stages  of  development  -  develop  plan,  approve,  then  shelve  for  later  use 

Warfighting  CINCs  will  make  requests  to  supporting  staff 
Need  to  ask  “what  effect  do  I  want  to  achieve?” 

Example  of  Kosovo  -  ran  out  of  CALCMs  -  more  dependent  on  technology 
Better  deliberate  planning  leads  to  a  better  operation 

HD/LD  assets 

Security  issues  (high  value). .  .reasons  why  we  fly  B-2s  from  Whiteman 
Political  issues 

We  have  problems  communicating  our  capabilities  and  doctrine 
Way  we  operated  spread  out,  compared  to  other  services,  did  not  effectively  show  true 
capabilities. 

We  had  this,  and  this,  and  that,  but  couldn’t  effectively  show 
AEF/AEW  in  effect  will  help  better  present  our  capabilities 
We  need  complete  force  visibility  for  the  concept  to  work 

Guard  and  Reserves!  Almost  all  AEF/AEW  deployments  will  require 
Guard/Reserve  call-up 

Could  be  a  trust  issue  -  Do  CINCs  have  confidence  in  the  platform? 

CINC  must  have  trust  and  education  of  capabilities  of  platforms 
Requires  complete  Operations  Environment  Research!  (Stage  I  of  planning) 

Described  through  the  flow  of  request/information  from  JFC  to  ACC 

What’s  already  in  theater  (pre-positioned  assets) 

Host  Nation  Support  -  what  will  they/what  can  they  support 
Lift  requirements 
Coalitions  forces 

Operations  vs.  Logistics  -  spread  out  assets  vs.  combined. 

Benefits  and  drawbacks  to  each 
Bottom  line  is  “personalities”  run  things 
People  go  on  experience 
Education  is  key 

Make  planners  aware  of  the  tools  and  platforms  we  have  available 
GCCS 

TARGET  -  COA  analysis 
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A  JOPES  specialty  course  -  held  at  Fort  Eustis,  VA 
DSAG  -  Deployment  Special  Action  Group  (J4  @  Joint  Staff) 

AF-Logistics  Management  Agency  -  Gunter 
DSN  596-4087/88 

SAAS  -  Studies  of  Advanced  Airpower 
In-depth  airpower  studies 
Affect  doctrine  -  smart  folks 

AF  Doctrine  Center 

Lack  of  standardization  of  “tools”  across  AF 
Purpose  of  IDS  (Integrated  Deployment  Systems) 

Integrates  MANPER/LOGMOD/DMES/COMPESS 
Deliberate  and  Crisis  Action  Planning  Systems 

Trust/Commander  Requirement 
Intra-service  information  flow 

All  of  this  requires  a  significant  Culture  Change! !  I 


END  OF  INTERVIEW 
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APPENDIX  I:  CENTAF  Confirmed  Value  Functions  With  Definitions 


MULTI-NATIONAL  COMPATIBILITY 


Multi-National  Compatibility 


1  CALCULATION  INFORMATION  | 

Percent 

Compabitility 

Score 

100 

1 

40 

0.5 

Rho 

-25 

0 

0 

Rho 

15 

Description:  Extent  to  which  host-nation  assets,  to  include  combat  aircraft,  equipment, 
spares,  and  support  equipment  can  be  incorporated  along  side  and  into  USAF  combat 
aircraft. 

Measurement:  “Percentage  Compatibility  of  Assets.”  0-100  percent.  At  100  percent 
compatibility,  the  staging  base  that  the  host  nation  maintains  and  operates  like  assets  for 
100  percent  of  the  assets  in  the  asset  set.  USAF  combat  aircraft  can  be  incorporated  into 
the  staging  base  without  bringing  in  testing  equipment,  spares,  or  excessive  maintenance 
equipment.  At  0  percent,  all  of  the  assets  in  the  asset  set  are  different  (no  like  assets 
between  United  States  Air  Force  combat  aircraft  and  host  nation  combat  aircraft).  In  this 
situation,  campaign  planners  would  need  to  bring  a  complete  supply  of  testing  equipment, 
spares,  and  maintenance  and  repair  equipment  for  all  combat  aircraft  brought  into  the 
staging  base.  The  entire  range  of  percentages  of  host-nation  compatibility  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 


153 


ABILITY  TO  RESUPPLY 


Ability  to  Resupply 


1  CALCULATION  INFORMATION 

Ratio 

Score 

0 

0 

0.2 

0.15 

Rho 

0.1 

0.6 

0.85 

1 

1 

Rho 

-0.1 

Description:  Continued  combat  operations  from  the  staging  base  are  dependent  upon  the 
ability  of  USAF  force’s  equipment  and  materials  to  be  resupplied.  Assuming  the  staging 
base  has  been  predetermined,  the  selection  of  combat  aircraft  assets  will  be  dependent 
upon  individual  asset’s  resupply  requirements.  Combat  aircraft  that  require  smaller 
amounts  of  POL,  munitions,  and  other  equipment  will  have  a  higher  inherent  value  than 
an  aircraft  type  that  requires  more  assets. 

Measurement:  “Ratio  of  Resupply  to  Consumption  to.”  0-1.  At  the  ratio  of  resupply 
to  consumption  increases  to  1,  US  force’s  ability  to  resupply  the  staging  base  with 
required  equipment  and  materials  (i.e.  munitions,  aircraft  replacement  parts)  for  an 
aircraft  asset  is  equal  to  the  consumption  of  those  same  equipment  and  materials.  As  the 
ratio  of  resupply  to  consumption  decreases  to  0,  combat  aircraft  consumption  rates  far 
exceed  our  force’s  ability  to  resupply  the  staging  base.  Ratios  above  1  would  not  be 
experienced,  as  they  imply  assets  are  being  shipped  to  the  staging  base  location  that  are 
not  needed.  The  entire  range  of  ratios  of  resupply  to  consumption  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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TOPOGRAPHY 


Topography 


Percent  of  Mission  Degraded 


CALCULATION  INFORMATION 


Topography 

Difficulties 


0 

1 


Score 


1 

0 


Rho 


-0.27 


Description:  The  extent  to  which  natural  and  man-made  land  formations  around  the  pre¬ 
determined  staging  base  effects  the  ability  of  individual  aircraft  assets  within  an  asset  set 
to  take-off  and  land  at  the  staging  base. 

Measurement:  “Degradation  of  Mission.”  0  -  100  percent.  At  100  percent  mission 
degradation,  no  sorties  can  fly  in  or  out  of  the  staging  base  due  to  topographical 
restrictions.  100  percent  mission  degradation,  is  a  theoretical  maximum,  because  if  all 
aircraft  assets  could  not  take-off  from  the  staging  base  with  enough  munitions  to 
successfully  accomplish  the  mission,  those  aircraft  would  be  directed  to  a  different 
staging  base.  At  0  percent  mission  degradation,  all  aircraft  can  fly  sorties  out  of  and 
return  to  the  staging  base  with  enough  munitions  and  fuel  to  successfully  complete  the 
required  aerospace  missions.  At  50  percent  mission  degradation,  combat  aircraft  can  fly 
into  and  out  of  the  staging  base  at  50  percent  capability.  50  percent  capability  is  defined 
as  an  aircraft  asset  that  can  only  be  configured  with  enough  fuel  and  munitions  to  achieve 
a  50  percent  probability  of  kill  over  the  enemy  target.  The  entire  range  of  percentages  of 
degradation  of  mission  due  to  staging  base  location  topography  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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FIGHTER  DISTANCE:  UNREFUELED 


1  CALCULATION  INFORMATION  1 

Unrefueled 

Miles  (1000)  from 
Target 

Score 

0 

1 

0.7 

1 

0.75 

0 

Description:  The  effectiveness  of  an  individual  asset  within  the  asset  set  is  dependent 
upon  the  distance  from  the  staging  base  to  the  target. 

Measurement:  “Distance  (miles)  From  Target.”  0  -  6000  miles.  This  measure  scores 
the  location  of  the  staging  base  from  the  general  enemy  target  area.  Bomber  and  fighter 
assets  have  different  distance  abilities;  therefore,  each  has  a  separate  curve.  Additionally, 
separate  curves  are  required  for  refueled  and  un-refueled  operations.  It  has  been 
determined  for  this  analysis  that  6000  miles  is  the  effective  distance  of  bomber  assets  due 
to  crew  capabilities.  The  effective  range  of  a  fighter  aircraft,  determined  for  this  analysis, 
is  2500  miles,  due  to  crew  capabilities.  For  each  type  of  aircraft  (fighter  or  bomber),  the 
entire  range  of  effective  distances  within  their  specific  ranges  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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FIGHTER  DISTANCE:  REFUELED 


BOMBER  DISTANCE:  UNREFUELED 


0  500  1000  1500  2000  2500  3000 

Unrefueled  Miles  From  Target 


1  CALCULATION  INFORMATION 

Unrefueled 

Miles  (1 000)  from 

Target 

Score 

0 

1 

1 

1 

2.5 

0  Rho  -0.1 

Description:  The  effectiveness  of  an  individual  asset  within  the  asset  set  is  dependent 
upon  the  distance  from  the  staging  base  to  the  target. 

Measurement:  “Distance  (miles)  From  Target.”  0  -  6000  miles.  This  measure  scores 
the  location  of  the  staging  base  from  the  general  enemy  target  area.  Bomber  and  fighter 
assets  have  different  distance  abilities;  therefore,  each  has  a  separate  curve.  Additionally, 
separate  curves  are  required  for  refueled  and  un-refueled  operations.  It  has  been 
determined  for  this  analysis  that  6000  miles  is  the  effective  distance  of  bomber  assets  due 
to  crew  capabilities.  The  effective  range  of  a  fighter  aircraft,  determined  for  this  analysis, 
is  2500  miles,  due  to  crew  capabilities.  For  each  type  of  aircraft  (fighter  or  bomber),  the 
entire  range  of  effective  distances  within  their  specific  ranges  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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BOMBER  DISTANCE:  REFUELED 


Bomber  Distance 


1  CALCULATION  INFORMATION  I 

Refueled 

Miles  (1000)  from 

Target 

Score 

0 

1 

2 

1 

6.5 

0  Rho  9.959581 

Description:  See  Bomber  Distance:  UnrefUeled 
Measurement:  See  Bomber  Distance:  Unrefueled 
Confirmed:  14  JAN  00  by  CENTAF 
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MUNITIONS  STORAGE  AREA:  FIGHTER  MUNITIONS 


Fighter  Munitions 


Percent  of  Fighter  Sorties  per  Day 


CALCULATION  INFORMATION 

% 

Sorties/Day  Score 
0  0 

1  1  Rho  -0.305 


Description:  Continued  fighter  operations  depend  heavily  on  the  pre-determined  staging 
base’s  ability  to  store  specific  munitions  for  fighter  operations.  Fighter  aircraft  can  only 
bring  one  load  of  munitions;  additional  sorties  require  the  ability  of  the  staging  base  to  re¬ 
load  the  fighter. 

Measurement:  “Fighter  Sorties  per  Day.”  0-100  percent.  At  100  percent  sortie  rate 
per  day,  the  staging  base’s  munitions  storage  area  (MSA),  (includes  storage  and  supply) 
can  support  100  percent  of  the  Commander  in  Chiefs  (CINCs)  fighter  sortie 
requirements  per  day.  At  0  percent  sortie  rate  per  day,  no  sorties  can  be  supported. 

While  0  percent  is  theoretically  possible,  0  percent  is  not  likely,  because  if  the  staging 
base  could  not  store  the  required  munitions  for  a  particular  fighter  sortie,  another  aircraft 
alternative  would  be  selected.  At  50  percent  sortie  rate  per  day,  only  50  percent  of  the 
sorties  required  of  the  particular  aircraft  asset  can  be  conducted.  The  entire  range  of 
percentages  of  fighter  sorties  per  day  supported  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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MUNITIONS  STORAGE  AREA;  BOMBER  MUNITIONS 


Bomber  Munitions 


Percentage  of  Bomber  Sorties  per  Day 


CALCULATION  INFORMATION 

% 

Sorties/Day  Score 
0  0 

1  1  Rho  -0.305 


Description:  Continued  bomber  operations  depend  on  the  pre-determined  staging  base’s 
ability  to  store  specific  munitions  for  bomber  operations.  Bomber  aircraft  can  only  bring 
one  load  of  munitions;  additional  sorties  require  the  ability  of  the  staging  base  to  re-load 
the  fighter. 

Measurement:  “Bomber  Sorties  per  Day.”  0-100  percent.  At  100  percent  sortie  rate 
per  day,  the  staging  base’s  munitions  storage  area  (MSA),  (includes  storage  and  supply) 
can  support  100  percent  of  the  Commander  in  Chiefs  (CINCs)  bomber  sortie 
requirements  per  day.  At  0  percent  sortie  rate  per  day,  no  sorties  can  be  supported. 

While  0  percent  is  theoretically  possible,  0  percent  is  not  likely,  because  if  the  staging 
base  could  not  store  the  required  munitions  for  a  particular  bomber  sortie,  the  bomber 
aircraft  would  re-locate  to  another  staging  base  or  another  aircraft  alternative  would  be 
used.  At  50  percent  sortie  rate  per  day,  only  50  percent  of  the  sorties  required  of  the 
particular  aircraft  asset  can  be  conducted.  The  entire  range  of  percentages  of  bomber 
sorties  per  day  supported  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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POL  STORAGE 


CALCULATION  INFORMATION 

Sortie 

Missions/Day  Score 

0  0 

1  1 


Description:  The  staging  base’s  POL  system  includes  storage  and  supply  from  both 
military  and  host-nation  contracted  sources.  POL  capability  for  a  particular  asset  set 
must  take  into  consideration  the  per-sortie  POL  requirements  of  each  individual  aircraft 
in  the  asset  set  as  well  as  the  daily  sortie  requirements  for  each  individual  aircraft. 

Measurement:  “Mission  Capability.”  0-100  percent.  At  100  percent  mission 
capability,  the  staging  base  POL  system  can  support  100  percent  of  the  Commander  in 
Chiefs  (CINC’s)  sortie  requirements,  whether  the  POL  comes  from  on-base  or  off-base 
contracted  sources.  At  0  percent  mission  capability,  no  sorties  can  be  supported.  While  0 
percent  is  theoretically  possible,  if  0  percent  capability  were  experienced,  aircraft  assets 
would  be  directed  to  a  different  staging  base.  The  entire  range  of  percentages  of  mission 
capability  due  to  POL  storage  and  supply  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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TANKER  SUPPORT  REQUIRED 


Tanker  Support  Required 


1  CALCULATION  INFORMATION! 

Tanker  Sortie 

Missions/Day  Score 

0 

1 

30 

0 

35 

0 

Description:  Depending  upon  the  location  of  the  staging  base  from  the  enemy  target  and 
the  fuel  requirements  per  aircraft  asset  sortie,  and  the  number  of  aircraft  assets 
conducting  sorties  at  a  particular  time,  multiple  tanker  assets  may  be  required. 

Measurement:  “Tankers  Required.”  0  -  10.  This  measure  scores  the  number  of  tankers 
required  in  the  theater  to  support  asset  set  sorties.  Depending  upon  the  fuel  requirements 
for  a  particular  aircraft  asset  and  the  distance  between  the  staging  base  and  enemy  target, 
tanker  assets  may  be  required.  Additionally,  the  intensity  of  combat  in  and  around  the 
enemy  target  may  require  tanker  support.  Note,  this  measure  only  counts  tanker  that  are 
specifically  dedicated  to  combat  operations;  cargo  and  intelligence  gathering  aircraft  are 
not  considered.  This  measure  counts  actual  numbers  of  tankers  required  over  the  area  of 
responsibility,  and  not  tanker  sorties.  No  tankers  required  (0)  would  be  experienced  if 
combat  operations  did  not  require  aerial  refueling.  Ten  tankers  required  (10)  would  be 
experienced  if  combat  operations  required  an  extensive  amount  of  sustained  aerial 
operations.  The  entire  range  of  number  of  tankers  required  is  possible,  although  10 
tankers  required  are  a  theoretical  maximum. 

Confirmed:  14  JAN  00  by  CENTAF 
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TOPOGRAPHY 


Topography 


Percent  of  Mission  Degraded 


CALCULATION  INFORMATION 

Topography 
Difficulties  Score 
0  1 

1  0  Rho  -0.27 


Description:  The  extent  to  which  natural  and  man-made  land  formations  around  the  pre¬ 
determined  staging  base  effects  the  ability  of  individual  aircraft  assets  within  an  asset  set 
to  take-off  and  land  at  the  staging  base. 

Measurement:  “Degradation  of  Mission.”  0  -  100  percent.  At  100  percent  mission 
degradation,  no  sorties  can  fly  in  or  out  of  the  staging  base  due  to  topographical 
restrictions.  100  percent  mission  degradation,  is  a  theoretical  maximum,  because  if  all 
aircraft  assets  could  not  take-off  from  the  staging  base  with  enough  munitions  to 
successfully  accomplish  the  mission,  those  aircraft  would  be  directed  to  a  different 
staging  base.  At  0  percent  mission  degradation,  all  aircraft  can  fly  sorties  out  of  and 
return  to  the  staging  base  with  enough  munitions  and  fuel  to  successfully  complete  the 
required  aerospace  missions.  At  50  percent  mission  degradation,  combat  aircraft  can  fly 
into  and  out  of  the  staging  base  at  50  percent  capability.  50  percent  capability  is  defined 
as  an  aircraft  asset  that  can  only  be  configured  with  enough  fuel  and  munitions  to  achieve 
a  50  percent  probability  of  kill  over  the  enemy  target.  The  entire  range  of  percentages  of 
degradation  of  mission  due  to  staging  base  location  topography  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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CLIMATE 


Climate 


Percent  degradation  of  Mission 


CALCULATION  INFORMATION 

Climate 

Affect  Score 
0  1 

1  0  Rho  -0.27 


Description:  The  extent  to  which  heat,  humidity,  adverse  wind  and  atmospheric  pressure 
would  affect  the  ability  of  particular  aircraft  assets  to  conduct  combat  operations. 

Measurement:  “Degradation  of  Mission.”  0  -  100  percent.  At  100  percent  Mission 
Degradation,  no  sorties  can  fly.  This  is  a  theoretical  maximum  degradation  of  mission, 
because  if,  for  instance,  the  cross  winds  at  the  staging  base  were  such  that  a  particular 
aircraft  asset  could  not  land,  that  aircraft  asset  would  be  directed  to  another  base.  At  0 
percent  Mission  Degradation,  all  sorties  (all  aircraft)  can  fly  at  100  percent  capability.  At 
50  percent  Mission  Degradation,  An  aircraft  asset  can  only  take-off  at  50  percent  cll 
sorties  (all  aircraft)  can  fly  at  50  percent  capability  (i.e.  only  50  percent  of  maximum 
munitions  loading  for  bombers).  The  entire  range  of  percentages  of  mission  degradation 
due  to  climate  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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ASSET  MISSION  FLEXIBILITY 


Asset  Mission  Flexibility 


Percent  Capability  of  All  Aerospace  Missions 


CALCULATION  INFORMATION 

Percent 

Mission  Score 
Capable 

0  0 

1  1 


Description:  Extent  to  which  the  aircraft  within  the  asset  set  can  perform  all  of  the 
possible  aerospace  missions  defined  in  AFM  1-1  and  AFDD  2-1.  The  relative  worth  of 
the  asset  set  would  be  high  if  that  asset  set  could  perform  any  mission  required  by  the 
theater  commander. 

Measurement:  “Percent  Capability  of  All  Aerospace  Missions.”  0-100  percent.  At  0 
percent  aerospace  mission  capability,  the  assets  in  the  asset  set  cannot  perform  any 
aerospace  combat  mission  required  by  the  theater  Commander  in  Chief  (CINC).  This  is  a 
theoretical  minimum,  as  all  aircraft  assets  are  capable  of  performing  at  least  one 
aerospace  mission.  At  100  percent  aerospace  mission  capability,  the  assets  in  the  asset 
set  can  perform  any  and  all  of  the  aerospace  combat  missions  that  could  possibly  be 
required  by  the  theater  CINC.  100  percent  aerospace  mission  capability  is  achieved  if  the 
assets  in  the  asset  set  can  collectively  perform  all  of  the  aerospace  combat  missions  as 
defined  in  Air  Force  Manual  1-1  and  Air  Force  Doctrine  Document  2-1.  The  entire  range 
of  percentages  of  aerospace  mission  capability  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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ASSET  SET  UTILIZATION 


Asset  Set  Utilization 


CALCULATION  INFORMATION 

Avg  Utility 

Score 

0 

1 

0.7 

0.98 

1 

0  Rho  -0.20766 

Description:  Extent  to  which  each  aircraft  in  the  asset  set  is  used  to  perform  aerospace 
missions  as  required  by  the  theater  commander.  This  assumes  that  aircraft  brought  into 
the  theater  can  be  used  to  their  operationally  safe  limit  at  a  rate  of  70  percent.  Utilization 
rates  higher  than  70  percent  can  be  damaging  to  the  aircraft. 

Measurement:  “Percent  Utility  of  All  Aircraft  in  Asset  Set.”  0-100  percent.  At  70 
percent  use  of  primary  aircraft  authorized,  all  assets  in  asset  set  are  being  used  to  their 
peak  operational  efficiency.  This  could  be  experienced  during  an  initial  surge  of  sortie 
activity,  where  USAF  combat  aircraft  have  been  tasked  to  halt  enemy  action.  At  100 
percent  used  of  primary  aircraft  authorized,  aircraft  in  asset  set  have  been  surged  to 
theoretical  maximum,  whereby  all  of  the  aircraft  in  the  asset  set  are  in  a  state  of  continual 
“turn  and  burn.”  100  percent  use  of  primary  aircraft  authorized  cannot  be  sustained,  as  it 
will  quickly  degrade  the  capability  of  the  aircraft  assets  to  perform  the  mission.  The 
entire  range  of  percentages  of  use  of  primary  aircraft  authorized  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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AVAILABILITY  OF  ASSETS 


Availability  of /^ets 


1  CALCULATION  INFORMATION 

MC  Rate 

% 

Score 

0 

0 

0.3 

0 

0.7 

0.5 

Rho 

-0.1 

1 

1 

Rho 

0.15 

Description:  Extent  to  which  aircraft  assets  in  the  asset  set  are  operationally  capable  of 
being  used  to  conduct  aerospace  missions  required  by  the  theater  commander. 

Measurement:  “Average  Mission  Capable  Rate  of  Assets  in  Set.”  0-100  percent.  At 
100  percent  average  mission  capable  rate,  all  of  the  assets  in  the  set  of  assets  have  a  100 
percent  mission  capable  rate.  This  would  mean  that  maintenance  and  repair  requirements 
for  each  individual  aircraft  in  the  asset  set  could  be  completed  without  reducing  the 
Mission  Capability  rate.  At  0  percent  average  mission  capable  rate,  all  assets  in  the  set  of 
assets  are  non-flyable.  This  should  be  a  theoretical  minimum,  because  under 
circumstances  where  all  of  the  assets  in  the  asset  set  were  grounded  due  to  maintenance, 
another  set  of  aircraft  would  be  chosen.  At  50  percent  average  mission  capable  rate, 
some  assets  may  have  a  mission  capable  rate  higher  than  50  percent,  while  some  assets 
may  have  a  mission  capable  rate  less  than  50  percent.  The  entire  range  of  percentages  of 
availability  of  assets  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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USAF  VS  ENEMY  CAPABILITY 


USAF  Vs.  Enemy  Capability 


1  CALCULATION  INFORMATION  I 

Ratio  Enemy  Capability 

to  Number  of 

Score 

Assets 

0 

0 

0.8 

0 

1 

0.2 

Rho  -0.03 

1.5 

0.5 

Rho  -0.22 

3 

1 

Rho  0.155 

Description:  This  factor  measures  comparison  of  enemy  assets  to  USAF  combat  aircraft 
assets.  Assuming  US  intelligence  can  determine  extent  of  enemy  capability,  measure 
shows  relative  worth  of  amount  of  aircraft  assets  brought  into  pre-determined  staging 
base.  One  issue  is  number  of  USAF  combat  aircraft  brought  into  staging  base;  another 
issue  is  capabilities  of  enemy  to  counter  USAF  combat  aircraft  brought  into  staging  base. 

Measurement:  “Ratio  of  Allied  to  Enemy  Forces.”  0-3.  As  ratio  goes  to  0,  enemy 
capability  vs.  capabilities  of  combat  aircraft  in  asset  set  increase  (i.e.,  capability  of  1 
enemy  MIG  may  equal  capabilities  of  3  US  F-15s).  As  ratio  goes  to  1,  capability  of 
enemy  forces  and  USAF  combat  aircraft  are  equal  (i.e.,  capability  of  1  enemy  MIG  may 
equal  capabilities  of  1  US  F-16).  As  ratio  goes  to  3,  capabilities  of  USAF  combat  aircraft 
exceed  capability  of  enemy  forces  (i.e.,  capability  of  3  enemy  MIGs  may  equal 
capabilities  of  1  F-1 17). 

Confirmed:  14  JAN  00  by  CENTAF 
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ENEMY  ISR 


Enerrv  ISR 


CALCULATION  INFORMATION 

Enemy  ISR  Score 
0  1 

1  0  Rho  -0.41 


Description:  This  measure  captures  the  enemy’s  ability  to  see  or  obtain  information  on 
US  activity  at  the  pre-determined  staging  base.  Enemy  ISR  could  be  obtained  through 
aerial  or  satellite  photography,  local  sympathizers  monitoring  US  activity,  or  spies  within 
the  host-nation  working  within  or  around  the  staging  base. 

Measurement:  “Activity  Exposure.”  0-100  percent.  At  0  percent  exposure,  the  enemy 
has  no  intelligence  on  or  ability  to  see  our  operations.  This  may  occur  due  to  the  enemy’s 
lack  of  ISR  capabilities  or  due  to  strict  secrecy  of  operational  activity  at  the  staging  base. 
At  100  percent  exposure,  the  enemy  has  full  knowledge  or  visibility  of  our  operations. 

100  percent  exposure  is  more  conceivable  than  0  percent  exposure,  as  the  presence  of  US 
combat  aircraft  at  a  staging  base  not  usually  hosting  US  forces  would  arouse  some 
suspicion  and  concern  of  enemy  leadership.  The  entire  range  of  percentages  of  activity 
exposure  is  possible,  due  to  the  capabilities  of  the  enemy  and  the  amount  of  local 
sympathy  for  the  enemy. 

Confirmed:  14  JAN  00  by  CENTAF 
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ENEMY  INTERFERENCE 


Enemy  Interference 


1  CALCULATION  INFORMATION  I 

Enemy 

Affect 

Score 

0 

1 

0.5 

0  Rho  -0.15236 

1 

0 

Description:  This  measure  captures  the  enemy’s  ability  to  interfere  with  US  operations  at 
the  pre-determined  staging  base.  Enemy  interference  is  any  action  on  the  part  of  the 
enemy  that  would  hamper  US  combat  operations.  Enemy  interference  could  range  from 
picketing  and  loitering  around  the  staging  base,  isolated  attempts  to  gain  access  into  the 
base  and  base  operations,  bomb  threats,  attacks  on  aircraft  taking  off  and  landing  at  the 
base,  or  attacks  on  the  base  itself 

Measurement:  “Operation  degradation.”  0-100  percent.  At  0  percent  operation 
degradation,  no  enemy  interference  of  any  kind  is  experienced  at  the  staging  base.  At 
100  percent  operation  degradation,  enemy  interference  has  halted  any  and  all  combat 
operations,  to  include  destruction  of  the  staging  base.  100  percent  operation  degradation 
is  a  theoretical  maximum,  because  if  an  actual  attack  on  the  staging  base  were  expected 
to  occur,  the  aircraft  would  be  re-located  to  a  different  staging  base.  The  entire  range  of 
percentages  of  operation  degradation  is  possible. 

Confirmed:  14  JAN  00  by  CENTAF 
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APPENDIX  J:  Delphi  Study  Subject  Matter  Expert  Pool 


HO/USAF 

Col  Robert  Allardice,  AF/XOPE 
Col  Wade  McRoberts,  AF/XOPS 
Lt  Col  Alan  Engler,  AF/XOOC 
Lt  Col  Kevin  Foley,  AF/XOPE 
Lt  Col  Mark  Anderson,  AF/XOPW 
Lt  Col  Jose  Rivera,  AF/XOPW 
Lt  Col  Michael  Davis,  AF/XOPX 
Maj  Karen  Kwiatkowski,  AF/XOPX 


CENTAF 

Mr  Phillip  Lamm,  69  CPS/DOXL 
Maj  Steve  Luxion,  69  CPS/DOX 
Lt  Col  William  Doneth,  69  CPS/DOXL 
Lt  Col  Gary  Rattray,  490SS/0STE; 


ACC 

Col  James  Ruth,  ACC/XOX 
Col  Stephen  E.  Wright,  ACC/XOD 
Lt  Col  Bill  Reynolds,  ACC/XOXC 
Lt  Col  Alex  Cruzmartinez,  ACC/XOX 
Maj  Larry  Hahn,  ACC/XOX 
Maj  Joe  Torres,  ACC/XOX 
Maj  Lavwence  Averbeck,  ACC/XOXD 
Capt.  Jennifer  J  Murphy,  ACC/XOXD 
Mr.  George  Nelson,  ACC/XOOS 
Mr.  Jeffrey  Williams,  ACC/XOOS 


AU 

Lt  Col  Q.  C.  Walters,  CADRE/WSJ 
Maj  Michael  Odowd,  CADRE 
Maj  Leletta  V.  Tatum,  CADRE/WSC 
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APPENDIX  K:  Comments  from  First  Delphi  Study  Questionnaire 


-  Vertical  hierarchy  ignores  the  significant  horizontal  inter-relationships  that  affect  the 
measures  of  merit  in  your  definitions 

-  I.e.  contextual  considerations  not  pertaining  to  beddown  location  may  make  a  more 
distant  location  more  effective 

-  Oversimplified  BOS  concerns;  Security,  Communications,  Civil  Engineering,  Water 
Source,  Billeting,  etc.  must  also  be  considered 

-  Staging  Base  Issues  should  be  considerations  under  Beddown  Location 

-  Place  beddown  location  higher  than  multi-national  compatibility  under  host-nation 
column 

-  Factors,  as  listed,  are  not  presented  in  priority  order 

-  Third-tier  ‘parent’  issues  are  not  addressed  in  their  priority  order 

-  Sub-issues  under  each  ‘parent’  issue  are  not  addressed  in  their  priority  order 

-  Remove  the  term  staging  base,  use  beddown  location 

-  Mowq  fighter /bomber  distance  and  ability  to  resupply  to  underneath  beddown  location 

-  Do  not  separate  fighter/bomber  munitions 

-  Due  to  theater,  not  just  beddown  location  requirements; 

-  Remove  the  term  Munitions  Storage  Area;  use  Munitions  stockpile  availability  and 
storage  facilities 

-  Remove  the  term  POL;  use  POL  stockpile,  storage  and  dispensing  capability 

-  Appears  correct  for  the  fighter/bomber  arena 

-  Not  suitable  for  application  to  a  peace  operation,  smaller  scale  contingency,  or  a 
humanitarian  assistance  operation 

-  Politics  and  Enemy  are  inseparable  and  should  be  reflected  as  related;  certainly  they  do 
not  stand  alone 

-  Need  better  definition  of  whose  intra-national  politics  is  being  discussed;  unclear  if  it 
is  between  US,  NATO,  UN,  or  enemy  country 

-  Need  to  add  a  block  for  “Enemy  Offensive  Counter  Air  Capability” 
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APPENDIX  L:  Background  paper  for  Delphi  Weighting  Interviews 


BACKGROUND  PAPER 
ON 

CAMPAIGN  SPECIFIC  ISSUE  WEIGHTING  INTERVIEW 


1 .  The  purpose  of  the  research  interview  is  to  obtain  opinions  and  knowledge  on  the 
relative  weightings  between  the  campaign  specific  issues  identified  during  previous 
interviews  and  research.  The  purpose  of  this  background  paper  is  to  provide  an 
understanding  of  the  process  of  the  interview  and  to  present  an  example  of  how  the 
interview  will  be  conducted. 

-  Previous  rounds  of  Delphi  study  have  determined  relative  ranking  of  Campaign 
Specific  Issues 

—  Identified  from  interviews  with  campaign  planners  from  HQ/AF,  ACC,  Ar 
University,  and  CENTAF 

~  Ranking,  based  on  level  of  importance,  of  issues  has  been  determined 

-  Interview  will  consist  of  comparing  15  sets  of  two  separate  campaign  specific  issues  to 
determine  relative  weight  of  all  issues 


2.  To  understanding  how  comparisons  will  be  conducted,  an  example  is  provided.  When 
deciding  to  purchase  a  car,  many  features  must  be  considered.  Consider  two  cars;  Car 
‘A’  has  all  3  of  the  safety  features  required  (i.e.  anti-lock  brakes,  traction  control,  and 
dual  air  bags),  and  costs  $25,000.  Car  ‘B’  has  1  of  the  3  safety  features  (dual  air  bags) 
needed,  and  costs  $20,000.  For  simplicity,  other  features  will  be  disregarded. 

-  First,  the  value  of  these  issues  (safety  features  available  and  cost)  based  upon  personal 
requirements  must  be  identified. 

—  Determine  ‘value’  for  the  price  of  the  car,  based  on  personal  requirements 

—  Paying  $20,000  or  less  is  optimal,  resulting  in  a  value  of  1.0  at  $20,000 

—  Paying  $30,000  or  more  is  bad,  resulting  in  a  value  of  0  at  $30,000 

—  Value  decreases  linearly  between  $20,000  and  $30,000  ($25,000  results  in  a  value 
ofO.5) 
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—  Determine  ‘value’  for  having  safety  features,  based  on  personal  requirements 

—  Having  all  3  safety  features  is  optimal,  resulting  in  a  value  of  1.0  for  all  three 

—  Having  none  of  the  safety  features  is  very  bad,  resulting  in  a  value  of  0. 

—  Value  decreases  equally  between  3  to  no  features  (1  feature  results  in  a  value  of 
0.33,  2  features  result  in  a  value  of  0.66) 

--  Using  the  example,  ‘Safety’  value  of  Car  ‘A’  is  1.0,  and  ‘Cost’  value  is  0.5 

--  ‘Safety’  value  of  Car  ‘B’  is  0.33,  ‘Cost’  value  is  1.0 

-  Next,  relative  weighting  between  two  issues  must  be  determined 

—  It  is  determined  that  safety  features  are  more  important  than  cost 

—  Compare  ‘range  of  measurement’  of  safety  (0  to  3  features)  to  ‘range  of 
measurement’  of  cost  ($20,000  to  $30,000) 

—  As  safety  features  more  important  than  cost,  compare  cost  to  safety  by  asking  the 
following  question: 

WHAT  PORTION  OF  THE  ‘RANGE  OF  MEASUREMENT’  OF  SAFETY 
EQUALS  TYiE  ‘RANGE  OF  MEASUREMENT’  OF  COST? 

—  Question  determines  level  of  concern  over  reduced  value  between  two  issues 

—  If  cost  swings  from  $20,000  to  $30,000  (best  value  to  worst  value),  what  would  be 
the  equivalent  loss  in  safety  features. 

—  Based  on  personal  belief,  it  is  determined  a  swing  in  cost  from  $20,000  to  $30,000 
(best  value  to  worst  value)  is  comparable  (same  level  of  unhappiness  is  felt)  to  a 
swing  in  losing  1  safety  feature  (going  from  3  to  2  safety  features) 

—  Although  we  will  not  be  discussing  this  in  the  interview,  value  analysis  of  the  two 
choices  continues  with  converting  range  of  measurement  to  value. 

—  It  is  determined  that  Safety  features  are  3  times  as  important  as  cost  (1  times  the 
entire  range  of  cost  =1/3  entire  range  of  safety) 
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~  Combining  both  issues  to  compare  different  cars,  individual  weights  must  sum  to  1 
‘Cost’  +  ‘Safety’  =  1 
3  X  ‘Cost’  =  ‘Safety’ 

Combining  two  equations,  ‘Cost’  +  (3  x  ‘Cost’)  =  1,  Therefore  Cost  =  0.25 

--  Weight  of ‘Cost’  is  0.25  and  weight  of ‘Safety’  is  0.75. 

-  Once  values  and  relative  weightings  identified  for  two  issues,  individual  cars  can  be 
compared  to  determine  ‘best  value’  car  for  the  buyer 

“  Comparison  of  different  cars  based  upon  summing  the  following  to  determine  total 
value: 

Value  of ‘Cost’  x  Weight  of ‘Cost’ 

Value  of ‘Safety’  x  Weight  of ‘Safety’ 

—  Using  example.  Car  ‘A’  commands  total  value  of; 

(0.5  X  0.25)  +  (1  X  0.75)  =  0.88 

—  Car  ‘B’  commands  total  value  of; 

(1.0  X  0.25)  +  (0.33  X  0.75)  =  0.50 

--  Therefore,  based  on  value  analysis,  choose  Car  ‘A’  as  car  with  highest  total  value 


3.  Over  the  course  of  this  research,  the  ranges  of  measurement  for  each  individual 
Campaign  Specific  Issue  has  been  determined.  The  relative  importance  ranking  of  the 
Campaign  Specific  Issues  have  also  been  identified.  The  purpose  of  the  interview  is  to 
obtain  the  relative  weightings  for  these  issues  in  the  manner  discussed  in  this  paper. 


4.  Thank  you  very  much  for  your  time  and  support  of  this  research  effort.  If  you  have 
any  questions  about  the  method  in  which  the  interview  will  be  conducted  or  for  any  other 
reason,  please  do  not  hesitate  to  e-mail  me  at  Christopher.Buzo@afit.af.mil. 


ILt  Christopher  Buzo/AFIT/ENV/(937)254-5895/9Feb00 
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